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Abstract 


An effective technique for solving optimization problems over massive data sets is to partition the 
data into smaller pieces, solve the problem on each piece and compute a representative solution from it, 
and finally obtain a solution inside the union of the representative solutions for all pieces. This technique 
can be captured via the concept of composable core-sets, and has been recently applied to solve diversity 
maximization problems as well as several clustering problems IMMi. However, for coverage and 
submodular maximization problems, impossibility bounds are known for this technique 0. In this pa¬ 
per, we focus on efficient construction of a randomized variant of composable core-sets where the above 
idea is applied on a random clustering of the data. We employ this technique for the coverage, mono¬ 
tone and non-monotone submodular maximization problems. Our results significantly improve upon the 
hardness results for non-randomized core-sets, and imply improved results for submodular maximization 
in a distributed and streaming settings. The effectiveness of this technique has been confirmed empiri¬ 
cally for several machine learning applications [25t], and our proof provides a theoretical foundation to 
this idea. 

In summary, we show that a simple greedy algorithm results in a 1/3-approximate randomized com¬ 
posable core-set for submodular maximization under a cardinality constraint. This is in contrast to a 
known O (k/2- j impossibility result for (non-randomized) composable core-set. Our result also ex¬ 
tends to non-monotone submodular functions, and leads to the first 2-round MapReduce-based constant- 
factor approximation algorithm with 0(n) total communication complexity for either monotone or non¬ 
monotone functions. Finally, using an improved analysis technique and a new algorithm PseudoGreedy, 
we present an improved 0.545-approximation algorithm for monotone submodular maximization, which 
is in turn the first MapReduce-based algorithm beating factor 1/2 in a constant number of rounds. 


*An extended abstract of this paper appeared at ACM Symposium on Theory of Computing (STOC)’ 2015. 


1 




1 Introduction 


An effective way of processing massive data is to first extract a compact representation of the data and then 
perform further processing only on the representation itself. This approach significantly reduces the cost 
of processing, communicating and storing the data, as the representation size can be much smaller than the 
size of the original data set. Typically, the representation provides a smooth tradeoff between its size and 
the representation accuracy. Examples of this approach include techniques such as sampling, sketching, 
(composable) core-sets and mergeable summaries. Among these techniques, the concept of composable 
core-sets has been employed in several distributed optimization models such as nearest neighbor search [10, 
and the streaming and MapReduce models [25, 9, 19, so. Roughly speaking, the main idea behind this 
technique is as follows: First partition the data into smaller parts. Then compute a representative solution, 
referred to as a core-set, from each part. Finally, obtain a solution by solving the optimization problem 
over the union of core-sets for all parts. While this technique has been successfully applied to diversity 
maximization and clustering problems 19, JJlj], for coverage and submodular maximization problems, 
impossibility bounds are known for this technique [19]. 

In this paper, we focus on efficient construction of a randomized variant of composable core-sets where 
the above idea is applied on a random clustering of the data. We employ this technique for the coverage, 
monotone and non-monotone submodular problems. Our results significantly improve upon the hardness 
results for non-randomized core-sets, and imply improved results for submodular maximization in a dis¬ 
tributed and streaming settings. The effectiveness of this technique has been confirmed empirically for 
several machine learning applications [25|], and our proof provides a theoretical foundation to this idea. Let 
us first define this concept, and then discuss its applications, and our results. 

1.1 Preliminaries 


Here, we discuss the formal problem definition, and the distributed model motivating it. 

Submodular Functions. We start by defining submodular functions 0. Let N be a ground set of items with 
cardinality n = |N|. Consider a set function / : 2 ri —>• M + U {0}. We say function / is monotone if for 
any two subsets X C Y C N, f{X) < f(Y). We say function / is submodular if and only if for any two 
subsets X C Y C N, and an item x <G N \ Y, we have the property of diminishing returns, i.e., 


f(X U {*}) - f(X) > f(Y U {x}) - f(Y). 


Given an integer size constraint k, we let f) ; be 

fk(S) = max f(S'). 

S'CS,\S'\<k 

The submodular maximization problem with a cardinality constraint is as follows: given a parameter k and a 
value oracle access to a non-negative submodular function / : 2 N —» If# 1 ' U {0}, find a subset S of cardinality 
at most k with the maximum value f(S). The most common algorithm for solving the above problem is 
algorithm Greedy which is as follows: start from an empty set 5 = 0, and in k iterations, find an item x 
with maximum marginal / value for 5 (i.e., x = argmax ygN /(5 U {?/}) — f(S)) and add this item x to 
5. We refer to this algorithm as algorithm Greedy and note that it is a (1 — ^-approximation for monotone 
sumodular maximization problem with a cardinality constraint. 

Randomized Composable Core-sets. In this paper, we assume that all n items of N do not fit on one 
machine, and we need to apply a distributed algorithm to solve submodular maximization problem. To deal 
with this issue, we consider distributing items of N into m machines with indices {1,... , m}, where each 
item goes to C randomly chosen machines. Let {T \, T 2 ,..., T m } be subsets of items going to machines 

1 While the concepts in this paper can be applied to other set functions, we focus on maximizing submodular set functions. 
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{1,2,..., m} respectively. In this case, we say that {7j . Tj..... T m } is a random clustering of N with 
multiplicity C, i.e., {Ti, T 2l ..., T m } is a family of subsets T) C N, where each item of N is assigned to C 
randomly chosen subsets in this family. Note that Tf s are not necessarily disjoint subsets of items. Only 
the case of C = 1 corresponds to a random partitioning of items into m disjoint parts. This case is the most 
natural way of applying this idea, and is studied in Section [2] As we see later, higher values of C can help 
us achieve better approximation factors (See Section H}. We are now ready to formally define randomized 
composable core-sets. 

Definition 1.1. Consider an algorithm ALG that given any subset T C N returns a subset ALG(T) C T 
with size at most If. Let {T |, T 2 ,..., T m } be a random clustering of N to m subsets with multiplicity C. We 
say that algorithm ALG is an algorithm that implements a-approximate randomized composable core-set of 
size k' with multiplicity C for / and cardinality constraint parameter k, if 

E [/*(ALG(Ti) U ... U ALG(T m ))] > a • E [f k (T\ U ... U T m )\ , 

where the expectation is taken over the random choice of {Xj, T 2 ,..., T m }. For brevity, instead of saying 
that ALG implements a composable core-set, we say that ALG is an a-approximate randomized composable 
core-set. 

For ease of notation, when it is clear from the context, we may drop the term composable, and refer to 
composable core-sets as core-sets. Throughout this paper, we discuss randomized composable core-sets for 
the submodular maximization problem with a cardinality constraint k. 

Distributed Approximation Algorithm. Note that we can use a randomized a-approximate composable 
core-set algorithm ALG to design the following simple distributed (1 — -)a-approximation algorithm for 
monotone submodular maximization: 

1. In the first phase, following the random clustering {Ti,..., T m } defined above, allocate items in N to 
m machines, i.e., machine i gets the subset Ti of items. 

2. Each machine i computes a randomized composable core-set S t C 7) of size k', i.e., S r = A LG (7) ) 
for each 1 < % < m. 

3. In the second phase, first collect the union of all core-sets, U = Ui <i< m Si, on one machine. Then ap¬ 
ply a post-processing (1 — i)-approximation algorithm (e.g., algorithm Greedy) to compute a solution 
S to the submodular maximization problem over the set U. Output S. 

It follows from the definition of the a-approximate randomized composable core-set that the above 
algorithm is a distributed (1 — -jo-approximation algorithm for submodular maximization problem. We 
refer to this two-phase algorithmic approach as the distributed algorithm, and the overall approximation 
factor of the distributed algorithm as the distributed approximation factor. For all our algorithms in this 
paper, in addition to presenting an algorithm that achieves an approximation factor a as a randomized 
composable core-set, we propose a post-processing algorithm for the second phase, and present an improved 
analysis that achieves much better than (1 — i)a-approximation as the distributed approximation factor. 

Note that the above algorithm can be implemented in a distributed manner only if k' is small enough 
such that mk! items can be processed on one machine. In all our results the size of the composable core-set, 
k', is a function of the cardinality constraint, k: In particular, in Section [2l we apply a composable core-set 
of size k! = k. In Section[4l we apply a composable core-set of size k' < 4 k, and as a result, achieve a better 
approximation factor. We call a core-set, a small-size core-set, if its size k' is less than k (See Section [5]). 
As we will see, the hardness results for small-size core-sets are much stronger than that of core-sets of size 
k or larger. 


3 


Non-randomized Composable Core-sets. The above definition for randomized composable core-sets is 
introduced in this paper. Prior work 10,51 define a non-randomized variant of composable core-sets 
where the above property holds for any (arbitrary) partitioning {Ti, T2, ..., T m } of data into m parts H, i.e., 
an algorithm ALG as described above is a a-approximate ( non-randomized) composable core-set of size k' 
for f, if for any cardinality constraint k, and any arbitrary partitioning {Ti, T2, ..., T m } of the items into m 
sets, we have / fc (ALG(Ti) U ... U ALG(T m )) > a ■ f k (T\ U ... U T m ). 


1.2 Applications and Motivations 

An a-approximate randomized composable core-set of size k' = 0(k) for a problem can be applied in three 
types of applications [19 1: (i) in distributed computation [16], where it implies an a-approximation in one 
or two rounds of MapReduces using the total communication complexity of 0{n), (ii) in the random-order 
streaming model, where it implies an a-approximation algorithm in one pass using sublinear memory, (iii) 
in a class of approximate nearest neighbor search problems, where it implies an a-approximation algorithm 
based on the locally sensitive hashing (under an assumption). Here, we discuss the application for the 
MapReduce and Streaming framework, and for details of the approximate nearest neighbor application, we 
refer to [19]. 

We first show how to use a randomized composable core-set of size ()(k) to design a distributed algo¬ 
rithm in one or two rounds of MapReduces 0 using linear total communication complexity: Let m = \Jn/k, 
and let (T ±,..., T m ) be a random partitioning where T t has s/kn items. In the distributed algorithm, we 
assume that the random partitioning is produced in one round of MapReduce where each of m reducers 
receives Ti as input, and produces a core-set Si for the next round. Alternatively, we may assume that the 
data (or the items) are distributed uniformly at random among machines, or similarity each of m mappers 
receives T, as input, and produces a core-set S r for the reducer. In either case, the produced core-sets are 
passed to a single reducer in the first or the second round. The total input to the reducer, i.e., the union of 
the core-sets, is of size at most mk' = ()(k) \Jrijk = 0{y/kn). The solution computed by the reducer for 
the union of the core-sets is, by definition, a good approximation to the original problem. It is easy to see 
that the total communication complexity of this algorithm is 0{n), and this computation can be performed 
in one or two rounds as formally defined in the MapReduce computation model [21]. 

Next, we elaborate on the application for a streaming computation model: In the random-order data 
stream model, a random sequence of n data points needs to be processed “on-the-fly” while using only lim¬ 
ited storage. An algorithm for a randomized composable core-set can be easily used to obtain an algorithm 
for this setting |18, 3|@. In particular, if a randomized composable core-set for a given problem has size k, 
we start by dividing the random stream of data into \Jn/k blocks of size s = \fnk. This way, each block 
will be a random subset of items. The algorithm then proceeds block by block. Each block is read and 
stored in the main memory, its core-set is computed and stored, and the block is deleted. At the end, the 
algorithm solves the problem for the union of the core-sets. The whole algorithm takes only Oisfkri) space. 
The storage can be reduced further by utilizing more than one level of compression, at the cost of increasing 
the approximation factor. 

Variants of the composable core-set technique have been applied for optimization under MapReduce 
framework |21, 24, 9j. 19, 0,53]. However, none of these previous results formally study the difference 
between randomized and non-randomized variants and in most cases, they employ non-randomized com¬ 
posable core-sets. Indyk et al. 53 observed that the idea of non-randomized composable core-sets cannot 


“It is not hard to see that for non-randomized composable core-set, the multiplicity parameter C is not relevant. 

3 These results assume k < n 1-e for a constant t. 

4 The straightforward way of applying the ideas will result in two rounds of MapReduce. However, if we assume that the data 
is originally sharded randomly and each part is in a single shard, and the memory for each machine is more than the size of each 
shard, then it can be implemented via one round of MapReduce computation. 

5 The paper s introduced this approach for the special case of fc-median clustering. More general formulation of this method 
with other applications appeared in y|]. 
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Problem 

Core-set Size 

R/N 

U/L 

Core-set Approx. Factor 

Distributed Approx. 

Mon. Submodular Max. 1191 

Poly (A:) 

N 

U 

°(^) 

- 

Mon. Submodular Max.* 

k 

R 

L 

1/3 

0.27 

Non-Mon. Submodular Max.* 

k 

R 

L 


max( , Jj > 0.14 

Mon. Submodular Max.* 

O(k) 

R 

L 

0.585 — O(jr) 

0.545-0(4) 

Mon. Submodular Max. 

poly (k) 

R 

U 


- 

Mon. Submodular Max. 

k' < k 

N 

UL 

©(I) 

0(t) 

Mon. Submodular Max. 

k! < k 

R 

UL 

eq/fj 



Table 1: This table summarizes our results. In the column titled ”R/N ”, ”R” corresponds to the random¬ 
ized core-set, and ”N” corresponds to the non-randonrized core-set notion. In the column titled ”U/L ”, 
”U” corresponds to an upper bound result, and ”L” corresponds to a lower bound result. The last column 
corresponds to the distributed approximation factor. All these results except the first row are the new results 
of this paper. Previously, no constant-factor approximation has been proved for a randomized composable 
core-set for this problem. See Section [L4l for comparison to previous approximation algorithms. The rows 
with a star - (*) are our most important results. 


be applied to the coverage maximization (or more generally subnrodular maximization) problems. In fact, 
all our hardness results also apply to a class of submodular m a xi miz ation problems known as the max¬ 
imum k-coverage problems, i.e., given a number k, and a family of subsets A C 2 X , find a subfamily 
of k subsets A±,..., whose union U j=\Aj is maximized. Solving max ^-coverage and submodular 
maximization in a distributed manner have attracted a significant amount of research over the last few 
years 11131. 1151 .124 ll ll. l23l. 1251. Other than the importance of these problems, one reason for the popular¬ 
ity of this problem in this context is the fact that its approximation algorithm is algorithm Greedy which is 
naturally sequential and it is hard to parallelize or implement in a distributed manner. 

1.3 Our Contributions 

Our results are summarized in Table 1. As our first result, we prove that a family of efficient algorithms in¬ 
cluding a variant of algorithm Greedy with a consistent tie-breaking rule leads to an almost 1/3-approximate 
randomized composable core-set of size k for any monotone submodular function and cardinality constraint 
k with multiplicity of 1 (see Section 0). This is in contrast to a known 0(-^j= i) hardness result for any 
(non-randonrized) composable core-set 0], and shows the advantage of using the randomization here. Fur¬ 
thermore, by constructing this randomized core-set and applying algorithm Greedy afterwards, we show a 
0.27 distributed approximation factor for the monotone submodular maximization problem in one or two 
rounds of MapReduces with a linear communication complexity. Previous results lead to algorithms with 
either much larger number of rounds of MapReduce ll 131.11 lh. and/or larger communication complexity [23]. 
This improvement is important, since the number of rounds of MapReduce computation and communica¬ 
tion complexity are the most important factors in determining the performance of a MapReduced-based 
algorithm [22]. The effectiveness of using this technique has been confirmed empirically by Mirzasoleinran 
et al [25] who studied a similar algorithm on a subclass of submodular maximization problems. How¬ 
ever, they only provide provable guarantees for a subclass of submodular functions satisfying a certain 
Lipchitz condition [25]. Our result not only works for monotone submodular functions, but also extends 
to non-monotone (non-negative) submodular functions, and leads to the first constant-round MapReduce- 
based constant-factor approximation algorithm for non-monotone submodular maximization (with 0(n) 
total communication complexity and approximation factor of 0.18). It also leads to the first constant-factor 
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approximation algorithm for non-monotone submodular maximization in a random-order streaming model 
in one pass with sublinear memory. 

Our next goal is to improve the approximation factor of the above algorithm for monotone submodular 
functions. To this end, we first observe that one cannot achieve a better than the 1/2 factor via core-sets 
of size k using algorithm Greedy or any algorithm in a family of local search algorithms. In Section |4j 
we show how to go beyond the 1/2-approximation by applying core-sets of size higher than k but still of 
size 0{k), and prove that algorithm Greedy with a consistent tie-breaking rule provides a 0.585-approximate 
randomized composable core-set of size k! < 4 k for our problem. We then present algorithm PseudoGreedy 
that can be applied as a post-processing step to design a distributed 0.545-approximation algorithm in one 
or two rounds of MapReduces, and with linear total communication complexity. For monotone submodular 
maximization, this result implies the first distributed approximation algorithm with approximation factor 
better than 1/2 that runs in a constant number of rounds. We achieve this approximation factor using one or 
two rounds of MapReduces and with the total communication complexity of 0(n). In addition, this result 
implies the first approximation algorithm beating the 1/2 factor for the random-order streaming model with 
constant number of passes on the data and sublinear memory. To complement this result, we first show that 
our analysis for algorithm Greedy is tight. Moreover, we show that it is information theoretically impossible 
to achieve an approximation factor better than 1 — 1/e using a core-set with size polynomial in k. 

Finally, we consider the construction of small-size core-sets, i.e., a core-set of size k! < k. Studying 
such core-sets is important particularly for cases with large parameter k, e.g., k = fl(n) or k = Q For 
our problem, we first observe a hardness bound of 0(¥) for non-randomized core-sets. On the other hand, 

k' 


in Subsection l5.21 we show an Q( >/ -approximate randomized composable core-set for this problem, and 


accompany this result by a matching hardness bound of 0(y jr) for randomized composable core-set. The 
hardness result is presented in Subsection l5.ll 

1.4 Other Related Work. 

Submodular Maximization in Streaming and MapReduce: Solving max ^-coverage and submodular 
maximization in a distributed manner have attracted a significant amount of research over the last few 


years llJ,ll5|, [24j,hjl, 23, 25, a. From theoretical point of view, for the coverage maximization problem, 
Chierchetti et al. 111311 present a (1 — 1/e)-approximation algorithm in polylogarithmic number of MapRe¬ 
duce rounds, and Belloch et al [11] improved this result and achieved log 2 n number of rounds. Recently, 
Kumar et al. 12311 present a (1 — 1/e)-approximation algorithm using a logarithmic number of rounds of 
MapReduces. They also derive (1/2 — e)-approximation algorithm that runs in O(^) number of rounds of 
MapReduce (for a constant 5), but this algorithm needs a log n blowup in the communication complexity. 
As observed in various empirical studies [22], the communication complexity and the number of MapRe¬ 
duce rounds are important factors in determining the performance of a MapReduce-based algorithm and a 
log n blowup in the communication complexity can play a crucial role in applicability of the algorithm in 
practice. Our algorithm on the other hand runs only in (one or) two rounds, and can run on any number 
of machines as long as they can store the data, i.e, it needs m machines each with memory proportional 
to ^ of the size of the input. One previous attempt to apply the idea of core-sets for submodular maxi¬ 
mization is by Indyk et al. 0] who rule out the applicability of non-randomized core-sets by showing a 
hardness bound of O ( ] -^=-) for non-randomized core-sets. The most relevant previous attempt in applying 


randomized core-sets to submodular maximization is by Mirzasoleiman et al ll25h . where the authors study 
a class of algorithms similar to the ones discussed here, and show the effectiveness of applying algorithm 
Greedy over a random partitioning empirically for several machine learning applications. The authors also 


6 For such large k, a core-set of size k may not be as useful since outputting the whole core-set may be impossible. For example, 
in the formal MapReduce model (Til l, outputting a core-set of size k for k = fi(n) is not feasible. 


6 






























prove theoretical guarantees for algorithm Greedy for special classes of submodular functions satisfying a 
certain Lipschitz condition [25] - Here, on the other hand, we present guaranteed approximation results for 
all monotone and non-monotone submodular functions. In fact, Ashwinkumar and Karbasi |@] observed 
that if one applies the greedy algorithm without a consistent tie-breaking rule, the approximation factor of 
the algorithm is not bounded for coverage functions. In this paper, we prove that a class of algorithms in¬ 
cluding greedy with a consistent tie-breaking rule provide a guaranteed 0.27-approximation algorithm for 
all monotone submodular functions. We also show how to achieve an improved 0.54-approximation using 
slightly larger core-sets of size O(k). Finally, there is a recent paper in the streaming model |@] in which 
the authors present a streaming 1/2-approximation algorithm with one pass and linear memory. Our results 
lead to improved results for random-order streaming model for monotone and non-monotone submodular 
maximization. 

Core-sets: The notion of core-sets has been introduced in |[3t]. In this paper, we use the term core¬ 
sets to refer to “composable core-sets” which was formally defined in a recent paper 0. This notion 
has been also implicitly used in Section 5 of Agarwal et al. |U] where the authors specify composability 
properties of e-kernels (a variant of core-sets). The notion of (composable) core-sets are also related to 
the concept of mergeable summaries that have been studied in the literature |Q]. As discussed before, the 
idea of using core-sets has been applied either explicitly or implicitly in the streaming model [18, 3] and 
in the MapReduce framework [21, 24, Moreover, notions similar to randomized core-sets have 

been studied for random-order streaming models [20]. Finally, the idea of random projection in performing 


algebraic projections can be viewed as a related topic 
composability over a random partitioning. 

1.5 More Notation 


1,0 0 but it does not discuss the concept of 


We end this section by presenting notations and definitions used in the rest of the paper. Let A(x, X) 

dcf 

denote the marginal / value of adding item x to set X, i.e., A(x,X) = f(X U {x}) — f{X). With this 
definition, the submodularity property is equivalent to saying that for any two subsets XC7CN and an 
item x € N \ Y, we have A(x, X) > A (x,Y). Consider the distributed algorithm described in Section [TT] 
For any random clustering {Tj,. .., T m }, we let S', (1 < i < m) be the output of algorithm ALG on set 7), 
i.e., Si = ALG (Tj). Also we let OPT be the optimum solution (arg max|gi <fc f(S)), and OPT 5 be the set 

of selected items of OPT in U”L 1 5j, i.e., OPT 5 c = OPT 0 (UT^Sj). Throughout this paper, we analyze a 
class of algorithms, referred to as /3- nice algorithms, with certain properties defined below: 


Definition 1.2. Consider a submodular set function f. Let ALG be an algorithm that given any subset T 
returns subset ALG(T) C T with size at most k!. We say that this Algorithm ALG is a /3- nice algorithm for 
function f and some parameter 3 iff for any set T and any item x <G T \ ALG(T) (item x is in set T but is 
not selected in the output of algorithm ALG), then the following two properties hold: 

• Set ALG (T \ { x }) is equal to ALG(T), i.e., intuitively the output of the algorithm should not depend 
on the items it does not select, and 

• A(x, ALG(T)) is at most . In other words, the marginal f value of any not-selected item 

cannot be more than (3 times the average contribution of selected items. 


2 Randomized Core-sets for Submodular Maximization 

In this section, we show that a family of /?-nice algorithms, introduced in Section [T31 leads to a constant- 
factor approximate randomized composable core-set, and a constant-factor distributed approximation al¬ 
gorithm for monotone and non-monotone submodular maximization problems with cardinality constraints. 
Later, in Subsection l2.21 we show that several efficient algorithms in the literature of submodular maximiza¬ 
tion are f3 -nice for some 3 € [1,1 + e] (for e = o(l)) including some variant of algorithm Greedy with 
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a consistent tie-breaking rule, and also an almost linear-time algorithm in J3], Before stating the theorem, 
we emphasize that in this section, we apply a composable core-set of multiplicity 1 which corresponds to a 
random partitioning of items into m disjoint pieces. 

Theorem 2.1. For any (3 > 0, any /3-nice algorithm A LG is a approximate randomized composable 

1_JL 

core-set of multiplicity 1 and size kfor the monotone, and -approximate for non-monotone submodular 
maximization problems with cardinality constraint k. 


Proof. We want to show that there exists a subset of Uf =l Si with size at most k, and at least an expected 
/ value of for monotone /, and ^ 1 / ? 2 , ^ OPT - > for non-monotone /. Toward this goal, we take the 

maximum of rnaxi<j< m /(.Sj) and /(OPT 5 ) as a candidate solution. We define some notation to simplify 
the rest of the proof. Consider an arbitrary permutation i r on items of OPT, and for each item x € OPT 
let OPT* be the set of items in ir that appear before x. We will first lower bound /(OPT 5 ) using the 
submodularity property in Lemma 12721 

Lemma 2.2. For any set of selected items, /(OPT 5 ) > /(OPT) — X^eOPTVu’? 1 Si) A (a?, OPT*) for a 
monotone or non-monotone submodular function f. 

Proof. First we note that /(OPT) — /(OPT 5 ) = £ xe opT\OPT s A(x,OPT* U OPT 5 ). Using submod¬ 
ularity property, we know that A(x,OPT* U OPT 5 ) is at most A(x,OPT*) because OPT* is a subset 
of OPT* U OPT 5 . Therefore /(OPT) — /(OPT n A) < X^eOPT\OPT s A(x, OPT*) which is equal to 
£xsOpt\(u^ 1 Si) A(x, OPT*) by definition of OPT 5 . This concludes the proof. □ 

Lemma lA2l suggests that we should upper bound )CzeOPT\(u^ s,) A(x. OPT*) which is done in the 
next lemma. 

Lemma 2.3. The sum £™ x ExsOpt nTAfc A ( x > 0pri E is at most & ( max i <i<m f(Si )) + ExeOPTnTAA 

(A(x, OPT*) — A(x, OPT* U Si)) for a monotone or non-monotone submodular function f. 

Proof. The sum £,-=i ExGOPTnTLS, A(x, OPT*) can be written as: 

m 

E E (A(s, OPT* U Si) + (A(x, OPT*) - A(x, OPT* U Si))) 
i= 1 xeOPTnTASi 

The first term in the sum is upper bounded by us j n g the second property of /3 -nice algorithms. To 

conclude the proof, we apply inequality /(S)) < maxi<,/<„, /(.S/), and use the fact that there are at most 
k items in OPT \ OPT s = U™ 1 (OPT 0 T) \ Sf). ~~ □ 

At this stage of the analysis, we use randomness of partition {Tj\f = x to upper bound the expected value 
of these differences in A values with the expected value of average of f(Si). This is stated in the following 
le mm a: 


Lemma 2.4. Assuming each item in OPT is assigned to a T, uniformly at random (which is the case for all 
items not only members of OPT ), we have that E[£^i ExeOPTnT i \S l A ( x > OPT*) — A(x, OPT* U Si)] 


is at most 


EE f(Si)} 


for a monotone submodular function f, and at most 


EEE f(Si)} + /(OPT) 


■ for a 


non-monotone subdmodular function f. 









Before proving the lemmas, we observe that putting these three lemmas together, we can finish the proof 
of the theorem. In particular, for a monotone or non-monotone submodular function /, we have that: 

E[/(0PT 5 )] > /(OPT) - /3E[ max /(S))] - E ^ =1 _ •^ 0PT ) 

l<i<m m rn 

> (1 - -)/(OPT) - (1 + m max /($)]. 

m 1 <i<m 

/-i_1_ \ r (r\DT\ 

This immediately implies that E[max{/(0PT 5 ), max!<j< m /(£,)}] > -—-, which concludes 

the proof for non-monotone functions. For monotone functions, we get the same proof except that we should 
exclude the — term. To complete the proof, it is sufficient to prove the lemmas. 

Proof of Lemma [2.4l The main part of the proof is to show that the sum of the A differences in the statement 
of the lemma is in expectation at most T- fraction of sum of A differences for a larger set of pairs (i. x). In 
particular, we show that 


m ^ m 

e E e A(x, OPT x ) — A(x, OPT 31 U S'*)] < — • E[^ A(x, OPT x ) - A(x, OPT x U Si)]. 

i =i xeOPTn Ti\Si m i =1 xGOPT 

To simplify the rest of the proof, let A be the left hand side of the above inequality, and B be its right 
hand side. Also to simplify expressions A and B, we introduce the following notation: For every item x and 
set TCff, let h(x, T) denote A(x, OPT x ) — A(x, OPT 1 U ALG(X’)). Also let 1 [x ^ ALG(T U {x})] be 
equal to one if x is not in set ALG(T U {x}), and zero otherwise. We note that A and B are both separable 
for different choices of item x and set T), and can be rewritten formally using the new notation as follows: 

m 

A - E E E Pr[Ti = T U {x}] 1 [x ^ ALG(T U {x})]h(x, T U {x}) 

i =1 a:GOPT TCN\{i} 
m 

B = E E E (. Pr[Ti = T U {x}}h{x, T U {x}) + Pr[Ti = T]h{x, T)) 

i= 1 ieOPTTCN\{i} 

m 

£ E E E l[x i ALG(T U {x})]h(x, T U {x}) (Pr[T 4 = T U {x}] + Pr [T t = T]) 

i =1 xGOPT TCN\{x} 


where the inequality is implied by the following simple observations. Function h is non-negative, so multi¬ 
plying the sum by l[x ^ ALG(T U {x})] (which is either zero or one) can only decrease its value. We also 
replace one h(x, T) with h(x, T U {x}) which does not change the value of the sum at all because when 
l[x ^ ALG(T U {x})] = 1 (its only non-zero value), ALG(T U {x}) is identical to ALG(T) using the first 
property of nice algorithms, and thus h(x, T) = h(x, T U {x}) by definition. So the inequality holds. 

Now we can compare A, and B as follows: For any set T C N \ {x}, we have Pr[Ti = T] and 

respectively. As a 


Pr[T t = T U{x}] are equal to Q) |T| (l - ^) |1 " 1 M 1 and (^) |J|+i (l - £) 

is equal to — which shows that A < B. 


-m 


JT1+1 


l-ITI-l 


result, the ratio 


Pr[Tj=Tj{x}] 


Pr[Ti=TU{x}]+Pr[Ti=T] 


11^ rv-'m f(Q.X I 

To complete the proof for monotone /, it suffices to prove that B < - ^ ,=1 - - ! , we note that for any i, 
X^eOPT A(x,O pTXu Si) = /(OPTU Si) —f (Si), and Xxsopt ^(x, OPT 21 ) = /(OPT). For a monotone 
/, we have /(OPT U S { ) > /(OPT) which shows that B < M , 

Although, we do not necessarily have /(OPT U Si) > /(OPT) for a non-monotone /, we show that 
Y^Li /(OPT U Si) > {m — l)/(OPT). For any 1 < * < m, we have that / (OPT U (u), =1 5j/)) + 
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/(OPT U Si+i) is at least /(OPT) + / (OPT U (u/L 1 , + ,SV)) by submodularity. Applying this inequality 
for each 1 < i < m, and using non-negativity of /, we imply that Y17= 1 /(OPT U Si) > (m - 1)/(OPT). 
Therefore B is at most —- + ( m ~( m ~^))/(° PT ) ^ anc [ t p e p rQ of is completed for non-monotone / as 


2.1 Overall Distributed Approximation Factor of /3-nice Algorithms 

In Theorem 12.11 we prove that if on each part (set Tf) of the partitioning, we run a /3-nice algorithm ALG, the 
union of output sets of ALG will contain a set of size at most k that preserves at least fraction of value 
of optimum set. If we run algorithm Greedy on the union of output sets U™, Si, using the classic analysis 
of algorithm Greedy, we can easily claim that the overall value of the output set at the end is at least 1 ^ L ^ e 
fraction of /(OPT). In Theorem l2.51 we show an improved distributed approximation factor. 


Theorem 2.5. Let S be the output of algorithm Greedy over U™,//, i.e. S = Greedy (U'/pS',) for a 
monotone submodular function f. Also let S be the output of non-monotone submodular maximization 
algorithm of Buchbinder et al. Mi on U"2 | S t when f is a non-monotone submodular function. The 

expected value o/max{/(5), max"L 1 {/(S'j)}} is at least for monotone and ^ 

for non-monotone submodular f. In particular, for (3 = l, the distributed approximation factors are > 0.27, 

and > 0.21 — for monotone and non-monotone f respectively. 

Proof. By applying lemmas [2T2l 12.31 and 12.41 we have that: /(OPT 5 ) > /(OPT) — ft max™ ) /(S',) — 
—• 1 Si) > /(OPT) — (1 + (3) max"/, /(£)) for a monotone submodular function /. Using the classic 

analysis of Greedy on submodular maximization in |26tl . one can prove that /(.S') > (1 - 1/ e)f (OPT 5 ) 
when / is monotone. By taking the expectation of the two sides of this inequality, and using Lemma l2~4l 
we have that: 


E[/(S)] > E 


(1 - 1/e) 


/(OPT) - (1 + (3) max f{Sf) 
1=1 


Bylettingp := E [“" lx { A^fmax^_ 1 {/(5 I )}}] , ^ a ^ ove i ne q ua iity implies that p > (1 —l/e)(l —(l+/3)p), 


P > i +( i-i/ 1 e / )(i+/3) - Consequently E[max{/(5), max™, {/(Si)}}] is at least ^^ 1 /^ 1 +p) 


and therefore 
which proves the claim for any monotone /. 


(1—1/ e )/(OPT) 


If / is non-monotone, we get a weaker inequality f(S ) > — -- by applying the algorithm of Buch- 


binder et al. [ 12]. Using lemmas [272112.3 1 and !2.41 we have that: /(OPT 5 ) > /(OPT) — f3 max™, /(Sj) — 
> ( 1 _i.)/(Qp T )-( 1+ /3) ma x"/ 1 f(Si). Similarly, we can claim that p > 1 ( ^ 1 ^y e . 

□ 


This implies the desired lower bound of on P- 

Non-monotone submodular maximization In Theorem 12. 11 we proved that for a non-monotone /, Greedy 
returns a (/ - yT-j-approximate randomized composable core-set. When we have a small number of ma¬ 
chines, m, this approximation factor becomes small. So we suggest the non-monotone submodular maxi¬ 
mization algorithm of Buchbinder et al. [12] as an alternative. We note that since the items are partitioned 
randomly, — fraction of optimum solution is sent to each machine in expectation, and the expected value 
E[/(OPTn Si)] is at least for any 1 < i < m. Using the algorithm of Buchbinder et al. we get a y-C- 

approximate randomized core-set. We also note that the 0.21 -approximation guarantee for non-monotone / 
in Theorem [23] holds for large enough number of machines m. For small m, this y-C approximation can be 
used as an alternative. 
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2.2 Examples of / 3 -Nice Algorithms 

In this section, we show that several existing algorithms for submodular maximization in the literature 
belong to the family of /3 -nice algorithms. 

Algorithm Greedy with Consistent Tie-breaking: First, we observe that algorithm Greedy is 1-nice if it 
has a consistent tie breaking rule: while selecting among the items with the same marginal value. Greedy can 
have a fixed strict total ordering (II) of the items, and among the set of items with the maximum marginal 
value chooses the one highest rank in II. The consistency of the tie breaking rule implies the first property 
of nice algorithms. To see the second property, first observe that (i) Greedy always adds an item with the 
maximum marginal / value, and (ii) using submodularity of /, the marginal / values are decreasing as more 
items are added to the selected items. Therefore, after k iterations, the marginal value of adding any other 
item, is less than each of the k marginal / values we achieved while adding the first k items. This implies 
the 2nd property, and concludes that Greedy with a consistent tie-breaking rule is a 1-nice algorithm. 

An almost linear-time (l+e)-nice Algorithm: Badanidiyuru and Vondrak |[§|] present an almost linear-time 
(1 — - — e)-approximation algorithm for monotone submodular maximization with a cardinality constraint. 
We observe that this algorithm is (1 + 2e)-nice. The algorithm is a relaxed version of Greedy where in each 
iteration, it adds an item with almost maximum marginal value (with at least 1 — e fraction of the maximum 
marginal). As a result, similar to the proof for Greedy, one can show that this linear-time algorithm y/^-nice 
and consequently (1 + 2e)-nice for e < 0.5. 

3 Hardness Results for Randomized Core-sets 

In Section [2j we showed that a family of (3 -nice algorithms are -approximate randomized core sets 
(e.g., tt- approximate for algorithm Greedy). Here we show what kinds of randomized core-sets are not 
achievable. In particular, we prove, in Theorem 13. II that if we restrict our attention to core-sets of size k, 
algorithm Greedy or any local search algorithm does not achieve an approximation factor better than | even 
if each item is sent to multiple machines (up to multiplicity C = o(y / m)). This leads to the following 
question: does increasing the output size of core-sets, k' , help with the approximation factor? In other 
words, can we get a better than 1/2 approximation factor if we allow the algorithm to select more than k 
items on each machine? To answer this question, we first prove, in Theorem 13.31 that it is not possible to 
achieve a randomized composable core-set of size k! = o(^A-) with approximation better than 1 — / even 
when we allow for multiplicity C = o(\fjz)- We then show in Section 0] that although it is not possible 
to beat the 1 — i barrier, we can slightly increase the output sizes, apply algorithm Greedy to achieve an 
approximation factor ~ 2 — s/2 > 3 with a constant multiplicity. 

Following we show a limitation on core-sets of size k. In particular, we introduce a family of instances 
for which algorithm Greedy and any algorithm that returns a locally optimum solution of size at most k do 
not achieve a better than / + e-approximate core-set for any e > 0. This lower-bound result applies to a 
coverage valuation (and therefore submodular) function / and it holds even if we send each item to multiple 
machines. 

Theorem 3.1. For any e > 0, assuming each item is sent to at least one random machine (to set T, for a 
random l < i < n), and at most C < sj -y- random machines, and the number of items an algorithm is 
allowed to return is at most k, there exists a family of instances for which algorithm Greedy and any other 
local search algorithm returns an at most (/ + ];+ e)-approximate composable core-set. 

Proof. Let N be a subfamily of subsets of a universe ground set U, and let / : 2 : [ —y R be defined as 
follows: for any set S C N, f(S) = | LUeS A\. Note that / is a coverage function, and thus it is a monotone 
submodular function. Let the universe U be {1,2, ■ ■ ■ . k 2 + (k — l) 2 }. The family N of subsets of U consists 
of two types of subsets: a) k sets { A h }/ = |, and b) kL sets B,,j for any 1 < i < k, and 1 <j< L for some 
large L > m In {^bk .)• Let ^ rst set -^l subset {1, 2, • • • , k 2 }. For each 2 < i < k, we define 
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A, to be the set { k 2 + (.k — 1 ){i — 2) + 1, k 2 + (k — l)(i — 2) + 2, • • • , k 2 + ( k — 1 )(i — 1)} with size 
k — 1. Therefore the first type of sets { A}J' =] form a partitioning of the universe, and therefore they are 
the optimum family of k sets with the maximum / value. For each 1 < i < k, all sets Bjj are equal to 
{(i — l)k + 1, (i — 1 )k + 2, • • • , ik} with size k. So all type B sets with the same i value (and different j 
values) are identical. 

We say a machine is a good machine if for each 1 < i < k, it receives at least a set Bjj for some 
1 < j < L, and we call it is a bad machine otherwise. At first, we show that the output of algorithm Greedy 
or any local search algorithm on a good machine that has not received set A\ is exactly one set Bjj for each 
1 < ?' < k, and nothing else. In other words, these algorithms do not return any of the sets A 2 , A 3 , • • • . 
unless they have a set A\ as part of their input, or they are running on a bad machine. It is not hard to see 
that if .4 1 is not part of the input each set Bjj has marginal value k if no other set B t ,y for j' f j has been 
selected before. On the other hand, the marginal value of each of the sets {A,} -' =2 is A: — 1. So Greedy or 
any local search algorithm does not select any of the sets { A, }J ' =2 unless some set B tJ has been selected for 
each 1 < i < k. The fact that the output sizes are limited to k implies that sets {A t }J ' =2 are not selected. 

Now it suffices to prove that most machines are good, and most sets in {Aj] k =2 are not sent to a machine 
that has set A\ as well. To prove this, we show that each machine is good with probability at least 1 — 

To see this, note that for each i, there are L identical sets Bjj, and the probability that a machine does not 
receive any of these L copies is at most (1 — -^) L < e _i / m < 7^-. So the probability that a machine is 
good is at least (1 — 2 ^Jk) k > 1 — Since each set is sent to at most C machines, for each 1 < i < k, we 
know that set Aj is sent to only good machines with probability at least (1 — > l — §. We also know 

that the probability that for each 2 < i < k, the probability of set Aj sharing a machine with set A\ is at 
most — < % since each set is sent to at most C random machines. As a result, at most % + § = e fraction of 
sets {Ai}\ =2 are selected by at least one machine in expectation, and therefore in expectation the size of the 
union of all selected sets (not only the best k of them) is at most k 2 + e(k — l ) 2 which is less than ^ ^ + e 

fraction of the value of the optimum k sets ({A ,}-' = ( ). □ 


Following, we prove that it is not possible to achieve a better than 1 - - + e approximation factor for 
submodular maximization subject to a cardinality constraint even if each item is sent to at most C machines 
where C < y/em, and each machine is allowed to return k' = items. We note that in this hardness 

result, the size of the output sets can be arbitrarily large in terms of k, i.e. for instance k' could be ( .l(2 k ) 
for some values of n,m,C, and k. This is an information theoretic hardness result that does not use any 
complexity theoretic assumption. In fact, the instance itself can be optimally solved on a single machine, 
but distributing the items among several machines makes it hard to preserve the optimum solution. Before 
presenting the hardness result, we state the following version of Chernoff bound (which we use in the proof) 
as given on page 267, Corollary A. 1.10 and Theorem A. 1.13 in 0]: 


Lemma 3.2. Suppose X\. .A 2 , • • • , X n are 0—1 random variables such that Pr\X t = 1] = p t , and let 
p = Yli= 1 Pi> an d -X” = J2i= 1 Xi. Then for any a > 0 


Pr[x — P > a] < e o-(a+A*)ln(l+a/A0 

Moreover for any a > 0 


Pr[X — p < —a] < e~ a2/fM 

Theorem 3.3. For any e > 0, /;: > 4 and C < , assuming each item is sent to at most C machines 

randomly, and each machine can output at most k! = c ^ (2 , c ^ x ^ items, there exists a family of instances 
for which no algorithm can guarantee a core-set of expected value more than (1 — 7 + e) fraction of the 
optimum solution. 
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Proof. S im ilar to the proof of Thcorcn ll. 1 1 we define N to be a family of subsets of the ground set U. We 
assume that |U| |N| = n, and also assume that |U| = kl for some integer l . For any set S C N, let 

f{S) = | U/i G 5 A\ which is number of items in the ground set U that are present in at least one of the sets of 
S. We define the n items N = {A \ , A 2 , ■ ■ ■ , A n } as follows. Each item Ai is a subset of U with size l. The 
first k items {.4, }-' =] form a random partitioning of the ground set U (they are k disjoint sets each with size 
l to cover all k£ = |U| elements of U). So these k items form an optimum solution with value |U| = kl. For 
each k < i < n, we define each item Ai to be a random subset of U with size l. 

Each item Ai (i < k ) of the optimum solution is a random subset of U with size £, so in a machine 4, 
cannot be distinguished from items not in the optimum solution unless some other item Aj (for some j < k) 
is present in the machine. Therefore any machine that receives only one item of the optimum solution will 


ij e^n/m 

return it with probability at most + e 3 


— 877 st; - Because each machine returns at most 


k! items, and with probability at least 1 — e 


e^n/m 


, it has at least (1 — e)— items. We note that since each 


item is sent to at least one machine, the average load is at least — > 2 in( 8 C/e) 

So among the optimum solution items that are alone in their machines, at most fraction of them will 
be selected. We note that each item is sent to at most C machines, therefore in total ^ optimum items will 
be selected in this category in expectation. 

On the other hand, the total number of optimum items that share a machine with some other optimum 


(Cfc ) 2 


< (an upper bound on the expected number of collisions). We conclude that in total 


item is at most 

at most y optimum items will be selected in expectation. Therefore the total value of the selected optimum 
items does not exceed | fraction of the optimum solution. 

It suffices to prove that any subset of k items from {Ai}™ =k+l do not have value more than 1 — ^ + | 
fraction of the optimum solution. Consider a subset of k items {A ai \ k =1 , where k < a, < n for any 
1 < i < k. Each element in the ground set U belongs to at least one of these sets with probability 1 — (1 — 
j,) k < 1 — \ + \ < 1 — 7 + |. The probability that there are more than (1 — ^ + §)|U| items in the union of 


K*-— , / 7 \ _ ._. 

these k sets is not more than e 3 < |e n " using Chemoff bound (Lemma l3.2l) . We note that 

we need |U| to be greater than V " ■■■■ ' ■ - We know that there are less than (^) < e nln ( fc ) size k subsets 

of {Afc+i, Ak+ 2 , ■ ■ ■ , A n \. Using union bound, we imply that with probability at least 1 — the union of 
any k sets in {Ak+\, Ak+ 2 , ■ ■ ■ , A n } has size at most (1 — i + |)|U|. We conclude that the expected value of 
the best core-set among the selected items is not more than ||U| + (1 —i + |)|U| + ||U| = (1 —^+e)|U|. □ 


4 Better Randomized Core-sets for Monotone Submodular Maximization 

In this section, we prove that although it is not possible to beat the 1 — ^ barrier, we can slightly increase 
the output sizes (to k! = (y/2 + 1 )k), and apply algorithm Greedy to achieve an approximation factor ~ 
2 — y/2 > \ with a constant multiplicity. Furthermore, we show in Theorem 14. 8 1 that our analysis is tight for 
algorithm Greedy even if we increase the core-set sizes significantly. Finally, we present in Subsection 14.11 
a post-processing algorithm PseudoGreedy that achieves an overall distributed approximation factor better 
than 1/2. In particular, after the first phase, we show how to find a size k subset of the union of selected 
items with expected value at least (0.545 — o(l))/(OPT). Since in this section, we are dealing with a 
monotone submodular function /, we can assume WLOG that /(0) = 0. 

Theorem 4.1. For any integer C > 1, any cardinality constraint k = o(m), algorithm Greedy is a 
^2 — y/2 — O (j: + j ^ -approximate randomized composable core-set of multiplicity C and size k' = 

(2\/2 + 1 )k for any monotone submodular function f. By letting C = K this leads to a randomized 
composable core-set of approximation factor 0.5857. 
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Proof. Let D = f 2s/2 + 1, and k' = Dk. Following our notation from Section [T31 let S r = f Greedy(Tj) for 
1 < i < m, where T, is the set of items sent to the machine i. Note that we can let \S- t \ = Dk, since if there 
are less than Dk items in 7/ WLOG we can assume the algorithm returns some extra dummy items just for 
the sake of analysis. 

Consider an item x € OPT. We say that x survives from machine i, if, when we send x to machine 
i in addition to items of T t , algorithm Greedy would choose this item x in its output of size k', i.e., if 
x € Greedy(T) U {.x})}. For the sake of analysis, we partition the optimum solution into two sets as 
follows: let OPTi be the set of items in the optimum solution that would survive the first machine, i.e., 
OPTi = f {x\x € OPT 0 Greedy(Ti U {x})}. Let OPT 2 = OPT \ OPTi, and h = |OPTi|, and 
k 2 = |OPT 2 | (note that ki + k 2 = k). We also define OPTj d = OPTi nOPT 5 , and OPT' = f OPTjUOPTs 
where OPT 5 is defined in Subsection 1 1.51 

We aim to prove that Ef/^U^Sj)] is at least ^2 — s/2 — O ^ + ln ^ j /(OPT).Since // is a mono¬ 
tone function, we have fk(S\ U OPTj) < /fe(U™j S/. Note that OPT^ is by definition a subset of 
OPT 5 , and consequently a subset of LlT/i S t . So it suffices to prove that E[/fe(5i U OPT^)] is at least 
^2 — s/2 — O ^ + ln ^E j ^ /(OPT). To do so, we first prove the following Lemma: 

Lemma 4.2. The expected value of set E[/(OPT')] is at least ^1 — 0 /(OPT). 

Proof. Let 7r be an arbitrary fixed permutation on items of OPT. For any item x € OPT, we define 7 t x to 
be all items that appear prior to x in 7 r. To prove the claim of this lemma, it suffices to show that /(OPT) — 
E[/(OPT')] is O - 1 /(OPT) j where the expectation is taken over the distribution of random clustering 

of items Equivalently, we can prove '^°E(opt) OPT ^ is O j. At first, we characterize both 

terms /(OPT), and /(OPT) — /(OPT') in terms of some A values. 

Claim 4.3. The optimum value /(OPT) is equal to Exgopt A(x, tt*). and the term /(OPT) — /(OPT') is 
at most ExgOptAopts A(x, ir x ). 

Proof. By definition of A values, we have that: ExgOpt A(x, n x ) = /(OPT) — /(0) = /(OPT). Sim¬ 
ilarly, we have that /(OPT') is equal to ExeoPT' A(x, tt x 0 OPT'). By submodularity, we know that 
A(x, ir x n OPT') is at least A(x, tt x ) because 7 t x 0 OPT' is a subset of tt x . Therefore, we have: 


/(OPT) -/(OPT') = Y, A ( x ^)~ E A(x, 7r x n OPT') 


xGOPT 


xsopt' 


< 


E A ( x ^ x )~ E A ( x ^ x ) 


ccEOPT 


xeOPV 


E &(*,**)= E A (^’) 


xGOPT\OPT' xGOPTi\OPT s 


where the last equality holds by definition of OPT'. 


□ 


Now, it suffices to upper bound the expected value of E[ExgopTi\opt s A(x, tt x )] by O (“Tw) /(OPT) 


as follows: 


E[ E A(x,7T*)] = E Pr[x € OPTi \ OPT 5 ] A(x, n x ) 


iGOPTi\OPT s 


xGOPT 
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We note that A(x, tt x ) is a fixed (non-random) term and therefore we can take it out of the expectation. 
Since /(OPT) is equal to X^xsopt A(x, 7 t x ), we just need to prove that Pr[x £ OPTi \ OPT 5 ] is at most 
O ^ ^ j for any item x £ OPT. 

We note that the first machine is just a random machine, and the distribution of set of items sent to it, 
Xj, is the same as any other set T, for any 2 < i < m. We consider two cases for an item x £ OPT: 

• The probability of x being chosen when added to a random machine is at most ln [ ( ?' 1 , i.e. Pr[x £ 
Greedy(Ti U {x})] = 

• The probability Pr[x £ Greedy (Tj U {x})] is at least ln ^ . 

In the first case, we know that Pr[x € OPTi] < -jp~, and therefore Pr[x € OPTi \ OPT 5 ] < ln ^ 
which concludes the proof. 

In the latter case, we prove that Pr[x OPT 5 ] = 0( ln ^ ) which implies the claim of the lemma as 
Pr[x £ OPT 5 ] > Pr[x £ OPT! \ OPT 5 ]. Let zi, z 2 , ■ ■ ■ ,ic be the indices of the C random machines 
where x is sent to, i.e. x € 7/ for 1 < 7 < C. Item x is not in OPT 5 if it is not selected in non of these C 
machines. Formally, Pr[x OPT 5 ] is equal to Pr[x ^ Greedy(T^), VI < 7 < C]. 

We note that for any 1 < i < m, and each item y £ N \ {x}, the probability Pr[y £ T,) is < —. 
Although any pair of sets T, and 7/ are correlated (where 1 < i,i' < m), the events x' £ 7) and x" £ T, 
are independent for any pair of distinct items x',x" £ N \ { x }. So for any 1 < 7 < C, the distribution 
of T it \ {x} is the same as Ti \ {:/;}. Therefore the probability that machine it selects item x, Pr[x £ 
Greedyf T/)], is equal to Pr[x £ Greedy(Ti U {x})]. Since we are considering the latter case, we imply that 
Pr[x € Greedy(Tj £ )] is at least ln ^ for any 1 < 7 < C. If the C sets {T^ \ {x}}^ =1 were not coiTelated 
(mutually independent), we could say that events x £ Greedy(T^) are independent. Consequently, we 

would have Pr[x ^ Greedy(X^), VI < 7 < C] is at most ^1 — ln ^- ^ ln ^ c ) = and therefore 

Pr[x £ OPTi \ OPT 5 ] < Y~, which completes the proof. Although these sets are coiTelated, we show that 
this correlation is bounded as follows. 

We start by constructing C sets {T' e }f =1 which are very similar to the sets {T , f }];/, with the exception 
that the sets {T' ( \ {x}}f =1 are mutually independent. We exploit this independence property to show that 
Pr[x Greedy(T/ ), VI < l < C] < Finally, we use a coupling technique to show that with high 
probability (1 — Oi ji)), the solution of algorithm Greedy on T lf is the same as its solution on 7/ for any 
1 < ( < C, i.e. Greedy(T^) = Greedy(T/ ) which completes the proof. Following we elaborate on the 
construction of sets {T- and how they are coupled with sets {T lf { . 

For any 1 < i < C, insert each item y £ N \ {x} in T' f with probability ( — independently. Insert 
x into all these C sets {T' if } ( ji =x . We note that for each item y £ N \ {x}. the C events y £ T'. are 
independent, and consequently sets { T' f \ {x}}]]/, are mutually independent. Since for each 1 < l < C, 
set T[ \ {x} has the same distribution as 7\ \ {x}, and also using the independence property, we have that 

Pr[x ^ U^Greedy (T- ( )\ < (1 — ) c < We are ready to explain the coupling technique. 

We show how to construct sets {T lf { from sets {T- e }f =1 - We start with 7=1, and increase it one 
by one. The first step is easy. We set T n to be equal to T! . For any 1 < i < C, we initialize T lf by 
T' ( , and we adjust T lf by a few item insertions and deletions to address the correlations between T lf and 
prior sets T n , T l2 , • • • . T lf _ ] . For any 1 < 7 < C, we also upper bound the size of T l( AT' f with high 
probability. For any y £ N \ {x}, if y has appeared a times in 7/ , T l2 , • • • , T l( _ 1 , it should be present in 
Ti e with probability ^ 7 /° instead of This probability is at most m _ C c+ i which is less than 7 ^ + 7 ^ 
for m > C 2 . So the presence probability of each item is not increased by more than from the initial 
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probability ^ = Pr[y £ P' ]. To increase the presence probability of y by some 5, it suffices to insert it to 
T lf with probability 1-( £-^— because after this extra probabilistic insertion, it will be in T lf with probability 

S + (! - Pi>T=UJ^ = £ + S - Therefore, we have that Pr[y £ T k \ T' e \ is at most 

On the other hand, the probability of y being present in P* £ should decrease only if y appears in T lf , for 
some 1 < £! < £. To reduce the probability of y being present in T lf by some <5, it suffices to delete it from 
Tj ( (when it is in T lf ) with probability Therefore, y is in T' f \ Ti e only if y is in intersection T lf , n T[ f 

for some £' < £. The probability Pr[y £ T lfl n T/J is equal to Pr[y £ T l( \Pr\y O P/J = £ x £ = ^ 
(note that sets Ti e ,, and T' lf are independent). Since there are at most C choices for £', the probability 
Pr[y £ T' ( \ T lf ] is upper bounded by 

By this coupling technique, we conclude that for any item y £ N\ { x }. the probability Pr[y £ T lf AT' ] 

is at most Since there are n items in total, and these events are independent for different items, we can 
use Chernoff bound lT2l o have: 

q/^3 

Pr[\T k ATl\ >4 + (C + 1 )^n] < e 4(1 - ln ( c )) < GT 2 

We also know that has expected size — > k, and by Chernoff bound 13.21 with probability at least 
1 — e -fc / 8 = 1 — 0(C~ 2 ), its size is not less than (note that C < k). For every £, we conclude that with 
probability 1 — O(^), we have: 


> 


n 

2m 


\Ti e AT'\ < 4 + (C + l)^Jn<4+—x - 
* ra z m m 


We are ready to prove that with probability Pr [G reedy (P^) = Greedy(T/ ), Vf < £ < C] is 1 — 0 (^). 
It suffices to show for every £ that Pr [Greedy (T le ) = Greedy (P/J] is 1 — 0 () using the Union bound 
for these C events. Since m is at least 12 C 6 Dk for constant value of C, with probability 1 — 0(^4), 

at most C ‘ 2 Dk • We note that T n \ T[ is a random subset of T lf , and its selection 


the ratio 


\ t h a %\ 

I PA 


IS 


has nothing to do with / values. Therefore the probability that P^ \ T- has some non-empty intersection 

| Ti.\T! |x|Greedy(7i,)| , 

with set Greedy(P^) is equal to - SrUl - ^ c 7 ' Assuming T, f \ P ( . has no common item 


\ t h\ 

with Greedy(T^), we can say that Greedy(P^ n T - f ) is the same as Greedy(P^) because removing some 
items from that algorithm Greedy does not select will not change the output of algorithm Greedy on 
it (it is the first property of (3 -nice algorithms explained in Definition II.21) . Formally, with probability 
1 — 0(^j), the intersection (T if \ T -) n Greedy(T^) is empty, and consequently, we have Greedy(P^) = 
Greedy (Pj £ \ (T t/ \ P' )) = Greedy (7^. n T' lf ). Similarly, we can prove that with probability 1 — O (, 
sets Greedy(P^), and Greedy(P M n T' f ) are the same. We conclude that with probability 1 — 0(^4), the 
sets Greedy(T^), and Greedy (T! ) are the same. Using union bound, we have this equality for all £ with 
probability 1 — O(^). We already know that Pr[x £ u£ (P/J is at least 1 — Therefore Pr[x £ OPT 5 ] = 
Pr[x £ U c =1 Ti e \ is 1 — O(^) which concludes the proof of lemma. □ 


Using Lemma 14,21 it is sufficient prove that ratio is at least 2 — \/2 — O (j ). In order 

to lower bound the ratio we write the following factor-revealing linear program LP k,k2 , and 

prove in Lemma l4~4l that the solution to this LP is a lower bound on the aforementioned ratio. 
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LP k,k 2 Minimize j3 Subject to: 


(1) (5 > 1 — a + a j + c j VJ C [Dk]lk\J\ = k 2 (4) 0 < a, dj,bj,Cj < 1 

^ — 1 

(2) aj + bj + Cj > -^—4—- VI < / < Dk (5) f3 > Ej=i a j 4" b j 4" c j 


VI < j < Dk 




(6) aj + bj + Cj > Oj + i + 6j + i + Cj + i VI < j < Dk 


f, (jS'-i UOPT') 

Lemma 4.4. For any integer k > 0, Vic ratio /(opt') /lV lower bounded by the optimum solution of 
linear program LP k,k2 for some integer 1 < ky < k. 


fh(Si UOPT / ) 

Proof. We want to prove that ratio /(o PT /) 1 is lower bounded by the solution of minimization linear 

program LP k,k2 . It suffices to construct one feasible solution with objective value f3 equal to ■ At 

first, we construct this solution for every instance of the problem, and then prove its feasibility in Claim 1431 
We remind that OPT' is the union of two disjoint sets OPT',, and OPT 2 . Fix a permutation 7r on the 
items of OPT' such that every item of OPTj appears before every item of OPT 2 in 7 r. In other words, 7r is 
an arbitrary permutation on items of OPT'| followed by an arbitrary permutation on items of OPT 9 . For any 
item x in OPT', define ir x to be the set of items in OPT' that appear prior to x in permutation it. For any 
1 < j < Dk, we define set S J to be the first j items of ,Sj . We set the linear program variables as follows: 


def 


P = 


def 

a — 


def 

a 1 — 


. def 

bj = 


def 
Cj = 


/fcQSiUOPT') 

/(OPT') 

ExgOPT 2 ^(x, 7T X ) 

/(OPT') 

ErgOPTa A ( X > nX U 5 '^ 1 ) “ A ( X ’ nX U E 

/(OPT') 

ExgOPTj A ( X > nX U “ A ( X > nX U SJ ) 

/ (OPT') 


/(ffl) - f(Si- 1 ) 

f (OPT') 


b J 


Claim 4.5. 


The above assignment forms a feasible solution ofLP k,k2 , and its solution is equal to 


fk(S lUOPTj) 
/(OPT') 


Proof. The claim on the objective value of the solution is evident by definition of j3. To prove that the 
constraints hold, we show some simple and useful facts about the marginal values of items in OPT. We note 
that ExeOPT' A ( x > nX ) = /(OPT') by definition of A values and the fact that /(0) = 0. Similarly for any 
1 < j < Dk, we have that: 


]T A(x, vr" U S j ) = /(OPT' U S j ) - f(S j ) (1) 

igOPT' 

We are ready to prove that all constraints (1), (2), • • • , (5) one by one. We start with constraint (1). For 
any set J C [DK] with size | J\ = /c 2 , we define S(J) to be {y 3 \ j € J} where yj is the jth item selected 
by algorithm Greedy in Si. We also define S'(J ) to be OPTj U S(J). Set S'(J ) is a subset of OPTj U Si 
with size at most £: 2 + k\ = k. Therefore /(S'( J)) is a lower bound on /ft(OPT', U Si). We can also lower 
bound /(S'( J)) as follows: 
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= y A (^ nX n 0PT i) + E A (%-’ 0PT i u (^ _1 n 5 ( J ))) 

rreoPT'j jeJ 

> Y A ( x > ’ rX ) + Yj A (yj) OPT 7 U 5 J_1 ) 

xgopt; jeJ 

(1 - a)/(OPT') + £ (/(OPT; U S?) - /(OPT; U S^ 1 )) 
j&J 

(l-a)/(OPT') + 

Y ~ /( 5J ' _1 ) + (/(° pT i u - (/(opt; u &- 1 ) - ns*- 1 ))) 

j&j v J 

(1 - a)/(OPT') + 

Y f(°i + b i + Cj)/(° pT ') + (/(opt; U S*) - /OS*')) - (/(opt; U ^- 1 ) - /(^- 1 ))^) 

jeJ V J 

The first equality holds by definition of A. The first inequality holds by submodularity of /, and knowing 
that 5 J_1 fl S(J) C S^- 1 . The second equality holds by definition of a, and the last equality holds by 
definition of Cj. We claim that (/(OPT; U S j ) — f(Si)j — (/(OPT; U Si- 1 ) — /(iv 7-1 )^ (which is 

part of the right hand side of the last equality) is equal to —bjf( OPT'). We note that (/(OPT; U Si) — 

is equal to (CxeOPT', A(x,n x U Si), and similarly (/(OPT; U S'- 7-1 ) — /(5' J ” 1 )^ is equal to 
Sxsopt' A ( x > 7 t x U Si -1 ). By taking the difference of them, we have: 

(/(opt; U Si) - f(S j )) - (/(opt; U Si- 1 ) - /(Si- 1 )) = Y A ( x ’ ^ u 5j ) ~ A ( x > ** u ^ 

/reOPT'j 

which is (by definition of bj) equal to —bjf( OPT'). We conclude that /(£"), and consequently fk(S\ U 
OPT;) are both at least (1 — a + Yljej a j + c j)f( OPT') which concludes the proof of constraint (1). 

We prove constraint (2) using the fact that algorithm Greedy selects the item with maximum marginal 
value in each step. We note that the right hand side of constraint (2) is a 3 + bj + Cj which is by definition 

the marginal gain of item yj divided by /(OPT'), i.e. A |(opt') We know that any item x € OPT 2 
will not be selected by algorithm Greedy if it is part of the input set which means that the marginal gain 
(aj + bj + Cj)/(OPT') = A (yj, Si- 1 ) is at least the marginal gain A(x. S :1 ~ 1 ) for any x € OPT 2 , and it 
is also greater than the average of these marginal gains. In other words, we have: 

, > ZLeoPT 2 A ( x > 1 ) _ Z^eoPT 2 A ( x ? & x ) 

a j+i+Cj_ |qp T2 |j(q PT /) “ k 2 f(OPV) 

To finish the proof of constraint (2), it suffices to prove the following inequality: 


Y H x ,s j - 1 )>f( OPT') 

x£OPT 2 



By definition of a, and a values, we have that: 


( 2 ) 
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3 - 1 

/(OPT') (a-X; 

j '=i 


j-i 


E A(x,7T x )-^ E 


xGOPT2 


j'=1 XGOPT 2 

j~ 1 


= ^ A (^ 7rX )- 2 E (a^/ljs 3 ''- 1 )- a^/usj' 

XGOPT 2 a;gOPT2j'=l 

= Y 1 A ^ x )- E ( a (^ x )- a (^u s ^- 1 )) 

XGOPT 2 XGOPT 2 

= e %^ u s ^ 1 ) 

XGOPT 2 

which completes the proof of Equation |2j and consequently constraint (2). 

Now, we prove that constraint (3) holds. By definition of a, and the fact that /(OPT') = Exgopt' 

X^goPT' A(x,ir x ) 

A ( 2 , ir x ), we know that 1 — a is equal to- f(OPT') -• We also know that: 


Dk 


E 6 i 


Efi1 Exsopt; a ( x , n x U5 J x ) - A(s, TT X U S 3 ) 

/(OPT') 

Exgopt; a <>> 7rX ) - A (ah ^ u sDfc ) ^ Exgopt', A (^ 
/(OPT') “ /(OPT') 


where the inequality holds because valuation function / is monotone, and therefore all A values are 
non-negative. This proves that constraint (3) holds. 

To prove constraint (4), we should show that variables aj, bj, Cj, and a are all in range [0,1] for any 

1 <j< DK. We know that /(OPT') is equal to ExgOpt' a ( :c - 7T ' x )■ Therefore by definition, a is equal to 

E A(rr 7 r x ) 

z goPT 2 ’ ^ Since OPT 2 is a subset of OPT', we imply that a is at most 1. We also know that A values 

■EicgoPT' z M 3: i 7r ) 

are all non-negative, and therefore a is non-negative. Now, we should prove that variables aj, bj, and Cj are 

all in range [0,1]. By definition, aj + bj + c 3 is equal to ^qpt') ^ — E^optE* = /(ofe) w h ere the 
inequality and equality are implied by the submodularity of /, and the fact /(0) = 0 respectively. If /({%}) 
is at least /(OPT'), the proof of Lemma I4i4l can be completed as follows. We know that fk(S\ U OPT)) > 
f ({'!)] }), an d therefore the ratio J ^ OPT q 1 is at least 1. On the other hand, there exists a very simple 
solution for LP k2,k with objective value f3 = 1 by just setting all variables to zero, and 3 equal to one which 
completes the proof in the case f({yj}) > /(OPT'). So we can focus on the case, f({yj}) < /(OPT') 
in which we have a :j + bj + Cj < -opt') — E So it suffices to prove that these three variables are non¬ 
negative to prove that constraint (4) holds. Variables aj and bj are non-negative because / is submodular, 
and S :l ~ 1 is a subset of S3 We use Equation[T|to prove non-negativity of Cj. By definition, aj + bj is equal 

to EzSOp T' A(X EoPT') ) " A(X ’ 7r " USJ) • applying Equation m we have: 


a j + bj 

This implies that: 


/(OPT' U S^- 1 ) - /{Si- 1 ) - /(OPT' U S 3 ) + f(S 3 ) 
/ (OPT') 
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f(S*) - fiSi- 1 ) - /(OPT' U S^ 1 ) + /(5 J ' _1 ) + /(OPT' U S*) - /(S') 

Cj ~ /(OPT') 

-/(OPT' U S^- 1 ) + /(OPT' U S'J) ^ 

- / (OPT') - 

where the last inequality holds because of monotonicity of /. 

We prove constraint (5) as follows. At first, we show that the right hand side of constraint (5) is simply 
equal to /|~Qp T q ■ We know that aj+bj+Cj = '^ SJ f\opv) — ~ for each I < j < k. By a telescopic summation, 

we have that the right hand side of constraint (5), Y2j =l a j + bj + Cj, is equal to ^^opff~ = ffopT') • 
definition of /3, and the fact that //,.(,S'i U OPT) j is at least f(S k ), we conclude that constraint (5) holds. 

To prove constraint ( 6 ), we note that by definition, aj + bj + Cj is A (yj, S r ' ). Since algorithm Greedy 
chooses the item with maximum marginal value at each step, we have A (yj, S'- 7 ' - 1 ) > A(y j+ i, S^- 1 ). By 
submodularity, we have A(j/j + i, S’- 7 ” 1 ) > A(y J+ i, S'- 7 ) = a ]+ \ + bj+\ + c 3+ \. We conclude that constraint 
( 6 ) holds. Therefore the proofs of Claim 14.Si and Lemma l4~4l are also complete. □ 

□ 


Finally, we show that the solution of LP k,k2 is at least 2 — \[2 — O(^) for any possible value of k 2 
which concludes the proof of Theorem 14.11 

Lemma 4.6. The optimum solution of linear program LP k,k2 is at least 2 — s/2 — 0(^)for any 0 < A: 2 < k. 


Proof. We consider two cases: a) k 2 < and b) k 2 > Jq- We first consider the former case which is 
easier to prove, and then focus on the latter case. If k 2 is at most j-, we prove that the objective function 
of linear program LP k,k2 (which is 3) cannot be less than 0.6 which concludes the proof of this lemma. 
Since all variables are non-negative, we can apply constraint (2) for each j in range [1, k], and imply that 

Ylj=i a j + ^ + Cj is at least ^1 — ^1 — ^ a. Using constraint (5), we know that ffl 3 =\ a j + bj + Cj is a 

lower bound for [3. We are also considering the case k 2 < yg, therefore 3 is at least ^1 — ^1 — 

(l — e~ 10 ) a > 0.9999a. If a is at least 0.586, the claim of Lemma l4~6l is proved. So we assume a is at 

dcf dcf 

most 0.586. We define three sets of indices: J\ — {1, 2, • • • , k 2 }, J 2 — {k 2 + 1, k 2 + 2, ■ ■ - , 2 ^ 2 }, and 

dcf 

J 3 = {2fc 2 + 1, 2k 2 + 2, • • • , 3 k 2 }■ We note that these sets have size k 2 , and therefore constraint (1) should 
hold for them. If there exists some set J C Dk with size k 2 such that YljeJ a j + c j ' s at least 0.3a, we can 
use constraint (1) to lower bound 3 with 1 — a + 0.3a = 1 — 0.7a > 2 — \/2. Therefore we can assume 
that Y^jeJi a :i + c j ' s at most 0.3a for i € {1, 2, 3}. Using constraint (2), we imply that: 



a j +^+ cj 

> 

0.7 a 

ieJi 



y, a i + bj + Cj 

> 

0.4a 

i&P 



a j + bj + Cj 

> 

0.1a 

j&Js 
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We also know that J\ U J 2 U J 3 C {1, 2, • • • , k }. We apply constraint (5) to imply that 3 is at least 
(0.7 + 0.4 + 0.1)a = 1.2a. This yields a stronger upper bound on a. If a is at least 2 y~^ < 0.49, the 
claim of Lemma 1431 is proved. So we assume a < 0.49. We follow the above argument one more time, and 
the proof is complete. If for some i € {1, 2,3}, the sum Ylj&Ji a j + c j * s at l east 0.16a, using constraint 
(1), we can lower bound /3 with 1 — 0.84a > 2 — y/2. Therefore we have YljeJi aj + Cj < 0.16a for each 
i € {1, 2 ,3} which yields the following stronger inequalities: 


y. a i + b i+°j 

> 

0.84a 

j&ji 



y a j + bj + c i 

> 

0.68a 




y a j + bj + Cj 

> 

0.52a 

3 &J 3 




By applying constraint (5), we have 3 > (0.84 + 0.68 + 0.52)a > 2a. We can also use constraint (1), 
and conclude that 3 > max{ 2 a, 1 — a} > 2 — s/2 which completes the proof for the former case k -2 < yg. 

In the rest of the proof, we consider the latter case &2 > yg. The structure of the proof is as follows: In 
Claim 14.71 we first show that without loss of generality, one can assume a special structure in an optimum 
solution of LP k,k2 , and then exploit this structure to show that any solution of LP k,k 2 is lower bounded by 
a simple system of two equations with some = 0{\) error. We can explicitly analyze this system of 

equations, and achieve a lower bound of 2 — y/2 on 3 (the key variable of the system of equations, and also 
the objective function of linear program LP k,k2 ). 

Claim 4.7, There exists an optimum solution for linear program LP k,k2 with the following three properties: 

• Cj = 0 for all 1 < j < Dk 

• aj > aj + ifor all 1 < j < Dk 

• constraint (2) is tight for all 1 < j < Dk 

Proof. It suffices to show that every optimum solution of LP k,k2 without changing the objective 3 can 
be transformed to a feasible solution with the above three properties. Consider an optimum solution 
(^3*■ a*, { a*. b*. c* } I j=\ j ■ We start by showing how c*s can be set to zero. Suppose c i > 0 for some 
1 < j < Dk. We can increase the value of a* by c*, and then set c* to zero. This update keeps a* + c* 
intact, and therefore does not change anything in constraints (1), (3), (5), and ( 6 ). It also makes it easier 
to satisfy constraint ( 2 ) since it (possibly) reduces the right hand side, and keeps the left hand side intact. 
Constraint (4) remains satisfied since a* + c* < 1 (otherwise 3 is a l so at least 1 which proves the claim of 
Lemma 1431 directly). Therefore we can assume c variables are equal to zero, and exclude them to have a 
simpler linear program: 
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LP k ’ k2 Minimize /3 

Subject to: 


(1) P > 1 -« + E j & J a 3 

VJ C [Dk\k\J\ = k 2 

v—' 7 — 1 

, X a ~ / J ■/ ! a 5' 


(2) a,j + bj > l 2 = 

VI < j < Dk 

(3) Ef=ibj<l~a 


(4) 0 < a, cij, bj < 1 

VI < j < Dk 

(5) P > Ej=i a j + b j 


(6) dj + bj > cij+i + bj +1 

VI < j < Dk 


Now we prove how to make a* variables monotone decreasing. Suppose for some j\ < j 2 , we have 
a*i = a* 2 — 25 for some positive 5. We set both variables a* t and a* 2 to their average, i.e. increase a* x by 
5, and decrease a* 2 by 5. 


* i £ 

a ji a h + 6 


a j2 ' 


<4 - s 


We also decrease b* t and increase b* Q by 5: 


J2 


b* := 
n 


b n - 5 


Now, we show that all constraints holds one by one. We note that it suffices to consider constraint (1) 
only for set J with maximum Ejg j °*j- We can assume that this set J with maximum t2jej a *j cannot 
contain j\ without having j 2 (either before of after the update) because a* f is at most a* 2 in both cases. 
Therefore, the right hand side of constraint (1) is intact, and it still holds. 

Constraints (2), (5), and ( 6 ) are all intact because a* + b* is invariant in this operation for any 1 <J< 

Dk. 

Constraint (3) holds because sum of b* variables remain the same. To prove constraint (4) holds, it 
suffices to show that all variables stay in range [0,1]. It is evident for a* values since we are setting them to 
their average. For b* values, we first prove that b* stays non-negative. We note that a* + bl > a* n + bt 

J J. J i. J J. 

using constraint ( 6 ). We also have a* n = a* 2 , and b* 2 > 0. Therefore, b* ± cannot be negative. To prove that 
b* 2 is (still) at most 1 , it suffices to note that the sum of all b* values is (still) at most 1 — a < 1 , and b* 
variables are all non-negative. Therefore all constraints are still valid after this operation. 

We prove that after a finite number of times (at most ()) of applying this operation, we reach a 
feasible solution with monotone non-increasing sequence of a* values. If we start with j i = 1, and do this 
operation for any pair (ji, j' 2 ) with a* u < a * 2 , after at most Dk — 1 steps, we reach a solution in which 
a\ > a* for any 1 < j < Dk. We continue the same process by increasing j\ one by one, and after at most 
( D ? A ) updates, we reach a sorted sequence of a* values. By monotonicity of a* values, we can simplify the 
linear program even further: 
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LP k ’ k2 Minimize (3 


Subject to: 

(1) P > 1 _ a + Y:% 1 

(2) a,- + bj > ^ J 

(3) Efii bj < 1 - a 

(4) 0 < ct, a,j, bj < 1 

(5) P > £jL i Oj + bj 

(6) Oj + bj > a,j +1 + fej+i 

(7) QU ^ ^7 + 1 


fl'} 


VI < j < Dk 
VI < j < Dk 
VI < j < Dk 




Now we prove that we can assume that constraint (2) is tight for all 1 < j < Dk. At first, we prove 
by contradiction that the right hand side of constraint (2) is non-negative. Let j be the minimum index for 
which the right hand side of constraint (2) is negative. We can set all a* and b* values to zero for any index 
greater than j, and also reduce a*_ 1 by some amount to make this right hand side zero. All constraints hold, 
and we will have a solution in which the right hand side of constraint (2) is always non-negative. Now 
we make constraint (2) always tight as follows. Let j\ be the maximum index in range [1, Dk], for which 
constraint (2) is loose by some 5 > 0. We update the variables as follows. If b* x is positive, we reduce it to 
max { 6 ^ — 6, 0}, and do not change any other variable. We note that in this case, all constraints still hold, 
and constraint (2) for all indices ji, j\ + 1, • • • , Dk is tight. 

If b* n is zero, we decrease aj by 5, and for any j 2 > j\ , we increase a*j 2 by ^(1 — -L)f 2 -ji-i. We 
prove that constraint (2) for all indices ii, ji + 1, • • • , Dk is tight, and all other constraints still hold after 
this update. By definition of S, constraint (2) is tight for index j\ after this update. Constraint (2) was tight 
before the update for ]2 > j\ because of the special choice of j\. We prove that the right and left hand 
sides of constraint (2) increased by the same amount for each j 2 > j\. The left hand side increased by 
— A.)-? 2- :/' 1-1 . We also know that the right hand side changed by: 


s-YV= 


j'=j i+i 


&(1 


_Lv'-fi - 1 

k 2 > 


S-IM 1 - (1 - e)"-’ 1 A <5(1 - 


k 2 < 

k2 


Therefore constraint (2) is tight for all j 2 > ji a hcr the update. Now we prove feasibility of the new 
solution. For constraint (1), we note that all increments of a* variables is less than ~ /^) r = 5. 

We also note that a* x is decreased by 6. So the right hand side of constraint (1) is decreased, and it remains 
feasible. We just showed that for j? > ji, constraint (2) is tight and therefore valid, and it is intact for 
smaller indices. Constraint (3) is also intact. To prove constraint (4), we should show that the new a* values 
are in range [0,1], Since constraint (2) is tight, and a is at most 1, these new a* values are all at most 1. 
Non-negativity of the right hand side of constraint (2) implies that these new values are all non-negative. 
Constraint (5) holds since its right hand side is only decreased. To prove constraint ( 6 ), we note that the right 
hand sides of constraint (2) is decreasing in j, and they are all tight for indices > j\ . Therefore constraint 
( 6 ) holds for j > j\. For j = j\ — 1, it clearly holds since we are decreasing aj , and consequently its 
right hand side. To prove constraint (7), we note that the increments in a* values is decreasing as j 2 > j\ 
increases. So constraint (7) remains feasible for j > j\ . For j = j\ , we note that lj } is zero. So using 
constraint ( 6 ), we can prove constraint (7) holds for j = j\ which completes the feasibility proof. Each 
time, we make these updates, the index j\ (the maximum index for which constraint ( 2 ) is loose) reduces by 
at least (I). Therefore after at most Dk operations, we have an optimum solution with all three properties 
of Claim HU □ 


Using Claim 14771 we can assume that the solution of LP k ' k2 is lower bounded by the next pp new ^ k M_ 
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We focus on lower bounding the solution of Lp new > k to j n the rest of the proof. 


Lpnew,k,k 2 Minimize /3 
Subject to: 


(1) 13 > 1 - a + Yl?=i aj 


. , a—> , a-r 


(2) aj + bj — k2 

VI < j < Dk 

(3) £?=1 bj < 1 - a 


(4) 0 < a, aj, bj < 1 

VI < j < Dk 

(5) aj + bj > a j+i + bj+i 

VI < j < Dk 

(6) aj > a j+ 1 

VI < j < Dk 


We note that we eliminated one of the lower bounds on f3. This only reduces the optimum solution of 
the linear program which is consistent with our approach. We also used Claim l4~7l to replace the inequality 
constraint (2) with an equality constraint. We also removed the c variables, and added the monotonicity 
constraint (6). In the rest of the proof, we introduce some notation, and show some extra structure in the 
optimum solution of This will help us lower bound the optimum solution by analyzing a system 

of two equations explicitly. 

def 

We start with proving the extra structure. Let t = ay 2 . We show that for any pair of indices 1 < ji < 
32 < Ay, either b u is zero or a n is equal to r. Let ji be the minimum index with bj 1 > 0 . If j i is at least 
Ay, the claim holds clearly. So we consider j\ < Ay. If Oji+i is equal to r, by monotonicity of a values, 
the claim is proved. So we define 6 > 0 to be minify, ctj 1+ i — r}. We increase a ?l , and Lq+i by 5, and 
5(1 — ^-) respectively. We also decrease both of a n +\, and bj 1 by 6. After these changes, constraints (1), 
(2), (3), (4), and (5) in Lp new > k ’ k 2 s t ill hold. In particular, we made the changes in this special way to 
make sure that constraint (2) still holds. Constraint (5) also holds since the right hand side of constraint (2) 
is decreasing in j. But the monotonicity constraint (6) may be violated for j = j 1 + 1. This happens if the 
new aj 1+ i is less than a ?l + 2 - In this case, we swap the variables a JX + \ and a J} + 2 - We also change bj 1+ 1 , 
and b JI+ 2 in a way that constraint (2) holds for both j = j\ + 1, and j = j\ + 2. Similarly, we have that 
all constraints (1), (2), • ■ ■ , (5) hold. But constraint (6) may be violated for j = j\ + 2. We continue doing 
this swap operation until constraint (6) holds as well. This will happen after at most k 2 swap operations, 
since the new a values are all at least r. Finally, we reach a feasible solution for pp new , k > k 2 j n which either 
one more a variable is equal to t or one more b variable is set to zero. Therefore after at most 2A: 2 updates, 
for any pair of indices 1 < j\ < j '2 < A; 2 , we have that either b n is zero or a J2 is equal to r. 

We also claim that for any j > Ay, we can assume either aj = r, or bj = 0. Otherwise, suppose j\ > Ay 
is the smallest index for which < T, and bji > 0 - We can increase a ?l by S = min{r — a ?1 , bj 1 }, and 
decrease bj 1 by 5. We note that since a ?1 + bj 1 is invariant, the monotonicity constraints (5) still holds. We 
need to prove constraint (6) for j = j\ — 1. It can be violated only if ayy-i is less than r, and is 
non-negative which contradicts the choice of j\. To prove feasibility, we only need to prove constraint (2) 
for j > j\. We make it hold by the following adjustments. We start by j = ;j\ + 1, and increase it one by 
one. If constraint (2) is loose by some e for index j, we decrease bj by min {bj, e}. We also decrease aj by 
max{e — bj, 0}. The sum aj + bj is reduced by e, and constraint (2) now holds for j. It is also clear that sum 
of b variables do not increase, and all other constraints still hold. With these adjustments for each j > j\ in 
the increasing order, we know the solution is feasible. We also have all the ingredients to characterize the 
optimum solution of Lp new ’ k to , and lower bound it. 

We can formalize this optimum solution in terms of a few parameters including r, (3, k, and Ay. At this 
final stage of the proof, we conclude with two lower bounds (system of two equations) on (3 and 1 — a in 
terms of these few parameters. Let t be the smallest index in range 1 < t < Ay with bt / 0. If such an index 
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does not exist, define t to be k^. Using Claim 14.71 we know that constraint (2) is tight, and bj = 0 for any 


j < t, we can inductively prove that aj = ' ) :> 1 for any j < t. Consequently, we have that ]Cj = i a j 

is equal to a( 1 — ( k ' 2 ,~ 1 ) / -1 ) > a(l — e~ r ) where r is defined to be V-!-. Therefore, constraint (1) implies 


— a f fca-l Yi-l 
k 2 ^ k 2 

t -1 

~W~) > - “v- - ) ' - —— - — 

that: /3 > 1 — a + (1 — e _r )a + (1 — r)k 2 T. This lower bound on /3 is the first inequality we wanted to 
prove. To achieve the second inequality (lower bound on 1 — a), we start by upper bounding the sum of a 
variables. 

We show that Y?j=i a j — a (1 — e ~ r + ) as f°U° ws - Since a variables are monotone and, constraint 


(2) is tight for j = 1, we have at < a\ = j*-. We also have that Y^j=i a j * s ec l ua l to all — (1 — 


We can upper bound ^1 — -jj'j 
ing sequence for k ’2 = 1 , 2 , • • •. 


k 2 — 1 


k 2 


t-1 


by e 1 as follows. We prove that (1 — 


k 2 — 1 


is a monotone decreas- 


1 ~lk) 


k 2 — 1 


1 


1 - 


1 


k 2 


1 - 


k 2 +1 

We also know that liiri / i . 2 ^ 00 ( 1 
Therefore, we have that 


i x l‘ q) 


k 2 +l 


k 2 —1 


/C2 + 1 ( ^2 1 

> -:- X 1 — 


k2 


kl 


~*k) 


k 2 — 1 


= e 1 . Therefore each term (1 


+ 1 

U 3 
^ 2 


> i 




k 2 — 1 


is at least e . 


E 1 

3= 1 


^ — 



which yields the desired upper bound on X^=i a j- 

Since aj is at most r for any j > t, constraint (2) implies that bj > 


a—a^l—e r 


simplify the calculation, let a' be a — a (1 — e r + ^ ) = a ( e 


k 2 


— T. To 


— ^ ). Summing up the lower bounds 


^minjDfc—t—!,£*} / q'-l T 


— r ) where (:* is the greatest integer 


on b values imply that: 1 — a > Ylf=i bj > X^=o \ — j^~ 

less than or equal to — — k 2 (we set it to zero if — — k-i is not positive). By this definition of l *, we make sure 
that the right hand side summands are non-negative, and therefore they will not weaken the inequality. The 
rest of the analysis is done in two cases. If i* < Dk — t — 1, by computing the sum, we achieve following 
inequality: 


1 - a > 


Dk 


bj > 


- - 
3 =1 


t + 1 

k-2 


, , t*r\ ^ a 1 - k 2 r 


a' — k 2 T — 


a' — k 2 r \ (a/ — A^t)' 


2 k 2 r 


We conclude that if j3* is the solution of linear program the following system of equations 

should have a solution with /3 = /3 *: 
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( 3 ) 


P > 

1 — a > 


1 — a + (1 — e r )a + (l 



2A 


r) A 


where A is defined to be for. We note that a , A, and r are the variables of the above system of two 
equations, and they should be in range [0,1]. We also note that k 2 is another variable which can be any 
positive integer. To simplify, we solve the following system of equations to eliminate k 2 : 


> 1 — a + (l — e r )a + (l —r)A (4) 

^ ( e~ r a — A)" 

- 2A 

It is easy to see that if system of equations [3] has a solution [8\ , oq. A i, r |, k 2 ) , system of equations [4] 
has the following solution: (f3 = j3\ + a = a\, A = Ai + = r\). Therefore it suffices to lower 

bound j3 in system of equations |4] Because the same lower bound plus the term = 0(%) holds for /3 in 
system of equations [3] 

By computing the partial derivatives, and considering boundary values, one can find the minimum j3 
for which the system of equations [4] has a valid solution. Its minimum occurs when r is zero, and the 
second inequality is tight. Therefore we have a = \J A(2 — A). We conclude that 8 is the minimum of 

1- V / A(2 - A) + A which is equal to 1-^(1 - + \f\) + (l~\J\) = 2-2^ = 2-y/2sv 0.5857 

and occurs at A = 1 — ~ 0.2928. 

In the other case, £* is greater than Dk — t — 1, and therefore a' is at least Dkr by definition of l*. Since 
t < k 2 , we can write the following lower bound on 1 — or. 


P 

l — a 


2k 


1 ~ a >J2 b i> 

3 = 1 


Dk — t 


a 1 — /c2r — 


(Dk — t — 1 )t 


D -1 , , , v 

> —— (a - k 2 r ) 


So we have a slightly different set of two inequalities in this case to lower bound (3. 


/3 > 1 —a + (l —e r )a + (l —r)A (5) 

1 — a > —-— (e~ r a — A) 

It is evident that both inequalities should be tight to minimize j3, and therefore a is equal to 1+ ^~/g-r 
where D' is So we can write f3 as a function of just A and r: 

/?(A ’ r) = 1 “ i + D'e- r 6 ~ r + (1 _ r)A 

To minimize j3, either A should be at one of its boundary values {0,1}, or the partial derivative jjj 
should be zero. For A = 1, (3 cannot be less than 2 — r — e~ r > 1 — A > 2 — \pl. For A = 0, we have 
1 — a > D'a' > D'a, so a is at most = y/2 — 1, and therefore /3 is at least 1 — a > 2 — \/2. The 
only case to consider is when = 0 which means 1+ e D ^ e - r should be equal to 1 — r with a unique solution 
r* = 0.71 ± 0.001. Therefore /3 is equal to 1 — (1 — r*)(l + A — A) = r* > 2 — y/2. We conclude that any 
feasible solution of linear program LP k ’ k2 has (3 at least 2 — \/2 — O ( j -,) which completes the proof. □ 
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We show in the following Theorem that the (2 — s/2) ~ 0.585 lower bound on the approximation ratio 
of the core-sets that Greedy finds is tight even if we allow the core-sets to be significantly large. 

Theorem 4.8. For any e > 0, and any core-set size k' > k, there are instances of monotone submodu- 
lar maximization problem with cardinality constraint k for which Greedy is at most a (2 — y/2 + O(e))- 
approximate randomized composable core-set even if each item is sent to C < y/em machines. 

Proof. Let N be a subfamily of subsets of a universe ground set U, and let / : 2 M —> It be defined as 
follows: for any set S C N, f(S) = | LUgs A\. Note that / is a coverage function, and thus it is a monotone 
submodular function. 

Our hardness instance is inspired by the solution of Equation [4] in the proof of Theorem 14. 11 We remind 
the solution (j3 = 2 — \/2, cr = \[\ , A = for = 1 — \/P) ■ Let k ,2 = k — 1 , and k 3 = \-y-~\ . For each 


1 < i < k 2 , and 1 < j < k 3 , let be a set of [ 7 ] arbitrary elements in universe. Let B' be a set of 
(1 + e)kk 3 \^—^\ arbitrary items of A f. We assume sets B t js, and set B' are all pairwise disjoint. Define 
set Ri to be the union (row i of matrix B of sets) for any 1 < i < liy. Similarly define set Cj to 


be the union U /i, ? (column j of matrix B of sets) for any 1 < j < k 3 . Define set Xj to be union of 
Cj, and max{l, ( k 3 — h> — ( j — 1)) [7] + 1 } arbitrary elements of If such that sets {Xj }*£_ , are pairwise 
disjoint. We note that there are enough elements in B' to make these sets disjoint because of Equation [4] 
Since we want sets Xj s to be disjoint we need at least k 3 + ~^ 2+1 ( A;3 — /C2 — (j — 1 )) [7] items in B'. 

Therefore \B'\ suffices to be as large as ^ 3 + ( fc 3 ~ fcz )(k 3 -fc 2 +i) x |-^-| < (a-\) _o_^-|~ e ). Using Equation 0] 
we have 1 — a > which completes the proof of the lower bound claim on size of B'. We also define 

k' singleton disjoint sets { Zp}/_ 1 where k! is the the number of items Greedy is allowed to return. Each Zp 
contains one element of universe, and is disjoint from all other sets defined above. 

The hardness instance consists of one copy of set B ', one copy of set If (for each 1 < i < k>>), 
10 m In {mk) copies of set Xj (for each 1 < j < kf), and 10 m In (mk) copies of set Zi (for each 1 < l < k!) 
where m is the number of machines. 

First, we prove that with high probability, each machine has at least one copy of Xj for each 1 < j < k 3 . 
We note that for each j, and each machine M, at least 10 ln(mA') copies of Xj is sent to M, and using 
Chernoff bound lT2l with probability at least 1 — 7777 at least one copy of Xj is sent to M. Using Union 
bound, with probability at least I — 7 J 7 , for each machine is receiving at least one copy of Xj for any j. 
Similarly we know that each machine has at least one copy of set Zp for any 1 < £ < k!. 

We also note that for any 1 < i < k^, with probability at least 1 — e, none of the at most C copies of set 
Ri shares a machine with one copy of If. There are at most C copies of If, and C copies of If, and the 
probability that two sets are sent to the same machine is f-. So with probability at most — < e, one copy 
of R{ and one copy of If are sent to the same machine. 

In a machine M that does not have any copy of B' , and has at least one copy of Xj (for any 1 < 
j < k 3 ), Greedy does not choose any copy of If. We prove this by contradiction. Suppose at iteration t, 
Greedy chooses some set Ri in this machine. Sets X\, X-p. ■ ■ ■ ,X t -i are the first selected sets by Greedy 
since Xj sets are disjoint and monotone decreasing in size. We claim that the marginal gain of set X t 
is larger than the marginal gain of Ri which shows the contradiction. Marginal gain of selecting X t is 
[ 7 I + (k 3 — k 2 — {t — 1)) [ 7 ] + 1. We also know that marginal gain of selecting any row set Ri is equal 

to ^ [ ff 1 ] — (t — 1) j [ 7 ]. Clearly the marginal gain of adding set X t is larger which completes the proof 

of the claim. So Greedy selects all sets {Xj}^ =l at first. We note that after choosing all these sets, marginal 
gain of adding any row set Ri is zero, and therefore they will not be selected because Greedy can always 
choose a set Zp to get a marginal value of at least 1 . 


27 






So the expected value of best size k subset of selected items (sets) is at most ek 2 \Ri\ + k + \B'\ + k\Cj\ 
for any i, j. Because each row set Ri is selected with probability at most e, and they all have the same size. 
There are at most k column sets present in any set of k selected sets. We also note that column sets Cj s 
have the same size. Set B' is also counted in the computation, and at most k singleton sets 2/s should be 
counted. Therefore the maximum value of size k subsets of selected sets is at most: 


ek2\Ri\ + k + \B'\ + k\Cj\ < kk 3 (e\^] + 1 + (1 + e)r^-^|) + k 2 \^\ 

< k 2 k 3 (l + j)(l + 0(e))(a + l + ^^ + -) 

k e e 

< fc 2 fc 3 (l + 0 (e)) 1 ~ Q + A 

e 

where the last inequality holds for k = Q(-). On the other hand, the optimum solution consists of 
k 2 = k — 1 row sets /?.,s, and set B' with value at least: 

We conclude that the expected value of maximum value size k subset of selected sets is upper bounded 
by A + 1 — a + O(e) = 2 — \/2 + 0(e) times the optimum solution /(OPT). □ 

4.1 Improved Distributed Approximation Algorithm 

We remind that in the first phase, each machine 1 < i < m runs algorithm Greedy on set T, with k' = 
Dk (where D is 2\/2 + l).By Theorem 14. 1 1 there exists a size k subset of selected items U™ , S t with 
expected value at least 0.585/(OPT), but we do not know how to find this set efficiently. If we apply 
algorithm Greedy again on U™ , .S', to select k items in total, we achieve a distributed approximation factor 
of (1 — I) x 0.585 ~ 0.37. In the following, we present a post-processing algorithm PseudoGreedy that 
achieves an overall distributed approximation factor better than 1/2. In particular, after the first phase, 
we show how to find a size k subset of the union of selected items U-T, S, with expected value at least 
(0.545 -o(l))/(OPT). 

Algorithm PseudoGreedy proceeds as follows: it first computes a family of candidate solutions of size 
k+0(l), and keeps the one candidate solution V with the maximum value. It then lets 5 to be a random size 
k subset of V, and returns S as the solution. These candidate solutions, denoted by S k ^j (for 1 k '2 k 

and 4 1 C {1, • • • , 8 }) in Algorithm [Q are computed as follows: We first enumerate all k possible values 
of 1 < k' 2 < k (this notation is used to be consistent with the proof of Theorem 14.1b . Then, by letting 
kj = k— k' 2 , and k 3 = 32 , we partition the first 8/;- 3 items in set Shinto eight subsets {A,/ }/ = |, each 

of size fc 3 . The next step of the algorithm proceeds as follows: for any I C {1, 2, • • • , 8 } with |/| < 4 + 
we initialize set S k ^ j with the union of all sets A t > where i' € I. Then for k\ + (4 — \I\)k 3 iterations, 
we search all items in U£i Si, and insert the item with the maximum marginal value to S k , 2 j. Roughly 
speaking, by starting from all these subsets, we ensure that the selected set hits enough number of items 
in OPT. The upper bound we enforce on |/| is to make sure that the number of iterations at this step is 
non-negative, i.e. k\ + (4 — \I\)k 3 > 0. Finally we define V to be the set S^j with the maximum / value, 
and return a random subset of size k of V as the output set S. 

1 We choose any machine instead of machine 1 , and since the clustering is done at random, the analysis goes through. 
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1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 


Input: A collection of m subsets {Si,..., S m }. 

Output: Set S C U™ 1 Si with |Sj < k. 

Ff- 0 ; 

forall the 1 < k' 2 < k do 

^3^32f4l; 
y , u y ■ 

rv > r\j n^2 5 

Partition the first 8k 3 items of Si into 8 sets {A,/ each of size ; 
forall the I C {1, 2, • • • ,8} wit/i |/| < 4 + ^ do 
Sk' 2 ,i Uj/ e /Aj/; 
for k\ + (4 — \I\)k 3 times do 

Find arg maXjQjm iS . A (a:, Sy 2 j), and insert it to Sy 2 j m , 

end 

if /(S fc2 , j)>f(V) then V <— S k2 j', 

end 

end 

S i — a random size k subset of V ; 

Algorithm 1: Algorithm PseudoGreedy 


Theorem 4.9. Algorithm PseudoGreedy returns a subset S with size at most k, and expected value at least 
(0.545 - 0 (\ + 12P))/(0PT). 

Proof. Algorithm PseudoGreedy returns set S, and we aim to prove that E[/(S)] is at at least 0.545(1 — 
0(% + 0( ln ^ , ' > )))/(0PT) as follows. We prove that E[/(S)] is at at least (1 — 0(% + 0(^^)))/(OPT) 
times the solution of the following linear program LP r for some 0 < r < 1. This is a constant size linear 
program, and therefore we can find its optimum solution for any r. Although there are infinite number of 
choices for r, and we cannot find the optimum solution of all of them numerically, we can discretize the 
interval [0,1], and consider a constant number of cases for r. In particular, we know that r € [j^- 77 ^ 7 " for 
some 1 < d < 160. In each of these cases, we have an accurate estimate of r, and based on these estimates, 
we change LP r in a way that does not violate feasibility of any solution (we should round up or down r in 
a way that maintains feasibility of constraint (1)). Solving LP r for all these 160 cases, we imply that its 
optimum solution is always above 0.545 with the minimum happening for r ~ {A. Therefore it suffices to 

show that for some r, the solution of linear program LP r times (1 — 0(j: + '"j/A )) is a lower bound on 

Ef/(g) 1 

/(OPT) • 


LP r Minimize d 


Subject to: 

(1) /? > (1 — e 


- 1 -F-T X ^ 

4 1 — r 


V/ C {1, 2, • • • , 8 }&|/| £4 + fc) 
( 2 ) a' e + l/ e + c! g > — e> 


)(1 a Yli'ei K) + Yhi'&i J2 


fee. 


2 » + Pf + do 


( 3 ) EfJi b' e < 1 - 

(4) 0 < a' e , b' e , c'f, 

(5) 0 < a < 1 


a 


VI < t < 256 
VI < i < 256 


We need to borrow many notations from proof of Theorem 14. 1 1 We remind that Tj, and Sj are the sets 
of items sent to the first machine and the set of items selected in this machine respectively, i.e., S\ is equal to 
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Greedy(Ti). We also remind that D = 2y/2 + 1, and k' = Dk. Note that we can assume |Si| = Dk, since 
if there are less than Dk items in Tj, WLOG we can assume the algorithm returns some extra dummy items 
just for the sake of analysis. Consider an item x € OPT. We say that x survives from machine 1, if, when we 
send x to machine 1 in addition to items of Tj, algorithm Greedy would choose this item x in its output of 
size k', i.e., if x € Greedy(Tj U {.x})}. For the sake of analysis, we partition the optimum solution into two 
sets as follows: let OPTi be the set of items in the optimum solution that would survive the first machine, 

/jpf _ Hpf Hpf 

i.e., OPTi = {x\x € OPT 0 Greedy^ U {x})}. Let OPT 2 = OPT \ OPTi, and k v = |OPTi|, and 
k 2 = |OPT 2 | (note that h + k 2 = k). We also define OPTj = f OPTi nOPT 5 , and OPT' = OPTjUOPT 2 
where OPT 5 is defined in Subsection 1 1.5 1 Using Lemma l4~2l (restated here), we know that set E[/(OPT')] 
is almost equal to /(OPT). Therefore OPT' can be used as the benchmark instead of OPT. In other words, 
assuming the following lemma, it suffices to prove that l) is at least (1 - 0(j.)) times the solution of 
LP r for some r. 

Lemma 4.10. The expected value of set OPT', E[/(OPT')], is at least ^1 — 0 ^ ln ^ ) ) /(OPT). 

Algorithm PseudoGreedy picks a random size k subset of set V, and returns it as the output set S. We 
prove that V does not have more than k + 0(1) items. We note that V is equal to set Sy / for some k 2 
and I. Therefore |Vj is equal to ks\I\ + k\ + (4 — \I\)k 3 = k\ + 4&3 < k\ + k 2 + 128 = k + 128. Let 
a be an arbitrary permutation on items of V. We have that f( V) = ff rr ( - A(x,o x ) where a x is the set 
of items in V that appear before x in permutation a. We also have that f(S) = J2 x eS A(x, ° x O &)• By 
submodularity, we know that YlxeS A(x, aX O S) i s at least s A(x, a x ). Therefore the ratio —j^yy is at 
least ^ w h ere the expectation is taken over the uniform distribution of all size k subsets 

of V. We note that Pr[x € S] for each x€UiSj|^>l — O(^). We conclude that E[/(5)] is at least 
(1 — 0(^))f(V). Applying Lennna l4~2l we know that to complete the proof of Theorem 14.91 it is sufficient 
to show is lower bounded by the solution of LP r for some 0 < r < 1. 

Since algorithm PseudoGreedy enumerates all k possible values of k 2 , in one of these trials, k 2 is equal 
to /c 2 = | OPT 2 1 - From now on, we focus on this specific value of k 2 . We remind that h> = 321"^]. Let I' 

def 

be the set of indices that maximizes Sk 2 ,i, i.e., I' = argmax /c ^ 2 8 t.&| 7 | <4+ N f(Sk 2 ,i)- Define V' be 

the set Sk 2 ,r- By definition, we have f(V') < f(V). So it suffices to show that f{V') is at least /(OPT') 
times the solution of LP r for some 0 < r < 1 as follows. We define r to be ,, , 4/l : i , . 

Before elaborating on how the linear program LP r lower bounds the ratio we need to explain 

this linear program in more details. Linear program LP r has 2 + 3 x 256 = 770 variables: a, /?, {«( }//(, 
{^}r=i> { c / : }f =1 ■ The objective function of LP r is variable 3. Set II, of indices is defined to be {32(7 — 
1) + 1,32(i — 1) + 2, • • • , 327} for any 1 < i < 8. We note that r, and |/| are not variables, and therefore 
constraint (1) like all other constraints is a linear inequality. 

At first, we remind the feasible solution of LP k k2 constructed in the proof of Lemma 14.41 and then 

f(y') 

construct a feasible solution for LP r with 3 equal to ^q PT /) as follows. Fix a permutation 7 r on the items 
of OPT' such that every item of OPTj appears before every item of OPT 2 in ir. In other words, ir is an 
arbitrary permutation on items of OPTj followed by an arbitrary permutation on items of OPT 2 . For any 
item x in OPT', define ir x to be the set of items in OPT' that appear prior to x in permutation it. We set a to 

be — xe /(opT') —■ For any 1 < j < 8 we define set S J to be the first j items of S 1 . Now we are ready 
to define variables {aj, b 3 , Cj }f=\ as follows. 
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E, g QPT 2 A(x, 7 r* U S'" 1 ) - A (x,n x U ffl) 
/(OPT') 


SxgopTj A(x, 7 r" U S'- 7-1 ) - A(x,n x U #) 
/(OPT') 


We let the variable c 3 be the marginal gain of item y 3 divided by /(OPT') minus a 3 + bj where y 3 is the 
jth item selected in Si, i.e., c 3 := A ^opj/) ^ — CLj — bj. 

f(V') 

We are ready to present a feasible solution for LP r . We set j3 to be We keep the same value for 


a = 


/(OPT') 


< £ < 256, we define: 


a £ 


def 


def 


I 128 

E ■ 

r&i< 


K = 


E 


/ 

c* 


def 


r&i< 


E 


Co 




It suffices to prove feasibility of this solution for linear program LP r . Using Lemma l4~4l we know a, b, 
and c variables with a, and some choice of j3 form a feasible solution for LP k,k2 . Therefore, we can use 
constraints of LP k,k2 to prove feasibility of this solution for LP r . Using constraint (4) of LP k,k2 , variable a 
is in range [0,1] which proves constraint (5) of LP r . We also know that a , 6 , and c variables are non-negative 
which implies that by definition o', 6', and c' variables are non-negative (constraint (4) of LP r ). Constraint 
(3) of LP r holds using constraint (3) of LP k,k2 and by definition of 6 '. We can also prove constraint (2) of 
LP r using the same constraint in LP k,k2 as follows. For each \j%g\(£ — 1) + 1 < j < \j^g\£, we know 

Oi ^ a 

that a,j + bj + Cj is at least- £ =1 3 . Summing up this inequality for all (-p|-] {£ — 1) + 1 < j < \ j^g\£, 

and by definition of a', and d, we have that: 


+ bp + c £ ~' > 


I 128 I 1 

E 


“ - Eg 


a j' 


•'==r fi 8 i(/ o-j 

1 128 I ^ 


E 


/ 

OL — 2Lf'=l a l’ 


iK4l(Li)+i 


hi 


> 


rap 

E 


T—2-U 

a f 


1 (<-!)+! 


&2 


7 / WAf / 

k 2 -| ct — )Ei'=i a j' > a ~ z^i'=i Q i' 

1 128 1 k 2 ~ 128 


We note that the last inequality holds assuming the numerator is non-negative. In case, the numerator 
is negative, the constraint (2) holds using non-negativity of a', 6 ', and d variables. So we just need to 
prove constraint (1) (which is the most important constraint) of LP r . We remind that p is defined to be 


f(V) _ 

/(opt') “ max rc{i, 2 ,- 

suffices to prove that: 


/(.%,/) 

,8}&|/|<4+£l /(OPT') 


jU- So for every I C {1, 2, 


, 8 } with size at most 4 + ||, 


it 
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f(S k2 ,i 


> 1-e 


— 1— 


4 1 — r 


/(OPT' 

We note that Sk 2 j consists of two sets of items: 


i'ei tes,/ 


+ EE EE ( a '(■ + 4 + 4) 

i'ei eeB ir 


• Set S’(J) with /1 A :3 items corresponding to sets {/// which are added in the first phase. In other 
words, S(I) is Li t , = U»/ e j U *^ ( ._ 1)fe3+1 {yj} where y 3 is the jth item of S’,. 

• fei + (4 — | /1) A::> items added greedily in the second phase. 

Similar to proof of Claim R31 we define S'(I) to be OPT', U S(I). To be consistent in notation, we 

def 

define J to be the indices of items in S(I), i.e. J — {j\yj € S(I) }. Using the same argument in proof of 
Claim 1431 we know that: 


f(S'(I)) > (1 —a)/(OPT0 + E>i + c,)/(OPT') 

j&J 

= (1 - «)/(OPT') + E E (4 + 4)/(°PT') (6) 

This means that there are | OPT', | < k\ items that can be added to S(I) in the second phase, and increase 
its value to f(S'(I)). Since we are adding k\ + (4 — \I\)k 3 items greedily in the second phase, our final 
value f(Sk 2 y) is at least: 


k i +( 4 -|r|)fc.i 

f{S k2 j) > f(S(I)) + (1 — e *1 )(f(S'(I))-f(S(I ))) 

= f(S(I)) + (1 - e - 1 -^x I ^)( /(iS /( / )) _ f{s{1))) ( 7 ) 

where the equality holds by definition of r. We can also lower bound f(S(I)) as follows. 


mi)) = EE w,si-'ns{i))> EE ac m- 1 ) 

jeS(i) jGS(i) 

= /(opto ee a 3+ b 3 + cj = /(opto EE EE 4+4+4 


(8) 


jes(i) 


i’ei teB,/ 


where the inequality holds by submodularity of /, and equalities hold by definition. By combining 
inequalities [ 6 j[7] and [ 8 ] we conclude that: 


J V ' i'aT Pan., ' 


i'elieB ,/ 


_l_(±dm XT r- 

— g 4 1—r 


1 - a - EE EE 4 

i’Gl l&Bf, 


This proves the feasibility of solution (/3, a, {a^, b^, ), and consequently completes the proof of 


Theorem 14.91 


□ 
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5 Small-size Core-sets for Submodular Maximization 

5.1 Hardness Results for Small-size Core-sets 

We start by presenting the hardness results for non-randomized core-sets. 

Theorem 5.1. Forany 1 < k' < k, the output ofany algorithm is at most an 0(q~)-approximate composable 
non-randomized core-set for the submodular maximization problem. 


Proof. The hardness instance consists of set A of k, and set B of the remaining items. We define the 
submodular function / : 2 Al>b —>• R as f(S) = |,S' n .4 for any subset S C A U B. Suppose the (non- 
randomized) partitioning, puts all items in A in one part, and distributes the rest of the items in other parts 
arbitrarily. Since at most k! items in each part can be selected as part of the core-set, no matter what sets 
are chosen in each part, any size k subset of the union of output sets cannot have / value more than k!. On 
other hand, set A is a size k subset with / value k which implies the ) hardness gap. □ 


Now we construct the following family of instances to achieve hardness results for randomized core-sets 
in the submodular maximization problem. 

Definition 5.2. We define instance I k,k ' of the submodular maximization problem as follows. Define T to be 
\_Vk'k\. For each 1 < i < k — T, we add an item that represents the set {?'}. For each k — T < i < k, we 
add LpJ identical items all representing the same set {?'}. The value of a subset S of items, f(S), is equal 
to the cardinality of the union of all sets the items in S represent. This is a coverage valuation function and 
subsequently monotone submodular. 


Theorem 5.3. For any 1 < k' < k, with m = @(p) machines, the output of any algorithm is at most an 

V' 


0(\/ 4 -)- approximate randomized composable core-set for the submodular maximization problem. 


Proof. We say item {?'} is alone in machine l if there is no other item with the same set {?'} in ma¬ 
chine i for any 1 < i < k, and 1 < l < m. Each item is sent to one of the m machines uni¬ 
formly at random. We also know that there are 0(m) copies of set {?'} for each k — F < i < k. 
Therefore Pr [Some item with set {i} is sent to machine l, and is alone in machine £} is Q(l) for any pair of 
k — T < i < k and 1 < l < m. So machine l receives in expectation Q(T) alone items for any 1 < £ < m. 
Since we can permute the elements {1, 2, • • • ,k} arbitrarily, there is no difference between alone items with 
i < k — T, and the ones with i > k — T. We also know that each machine can output at most k' items. So 
the probability that an arbitrary set {i} for some 1 < i < k — Tis selected in its machine is at most 0( p-). 

We conclude that the union of all selected sets have size at most T + 0(^)(/,: — T) which is 0{\J~^k) by 
definition of T. It is easy to see that the optimum solution has value k as there exist k disjoint sets in this 
instance. So the approximation factor of any algorithm for instance I k,k ' is OU jr). □ 


5.2 y )-approximate Core-sets for Monotone Submodular Maximization 


In Theorem 15.31 we proved that it is not possible to achieve better than ()(-approximate randomized 
composable core-sets for the submodular maximization problem. Here we show that this bound is tight. We 
prove this by applying a randomized algorithm (which is different from algorithm Greedy): 


1. Let set Tt be the set of items sent to machine 1 < £ < m (each item is sent to one of the machines 
uniformly at random). First run algorithm Greedy on set 7) to select set ,Sj of k items. Note that Sp 
has k items, and not k’. 
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2. Let Tt to be -4=. Now, construct set S( C Si as follows. 

• Start with S' = 0, and while there exists an x €= Si with A(x, S() > insert x to S',. 

3. With probability 1/2 output a random size k! subset of SV, and with probability 1/2, return a random 
size k' subset of SL 

Theorem 5.4. For any 1 < k' < k, and rn > tj machines, the union of output sets by the above algorithm 


form an O 


-approximate randomized core-set for the submodular maximization problem. 


Proof. For any 1 < t < m, let S” be the output set (with size If) of machine t. First, we prove that 
E[,//(U// I S/)]is f l ( ^ e= m (S ^ )■ Let m' = |_pj < m. We choose a set M' = ,4n'} of 

m' uniformly at random machines. Let (/,; be the union of output sets of machines lx.fi-x ■ ■ ■ 1% for any 
0 < i < m! , i.e., Ui — 1 U* =1 S'" . We note that U m > has size k'm' < k 'p = k. Therefore /((/„,/) is at most 

Following, we show that E[/((7 m /)] > '^ ,e=1 ^ S ^ ■ For any i, we prove that E [f(Ui) — f(Ui-f)\ is at 
where 1 < i < m!. We know that adding the whole set Sf t to L/_ i has marginal 


least 


2k 


value at least f(Sg i ) — /((/,_ i). With probability 1/2, output set 5/ consists of k! random items out of the 
k items of ,S/, ( . By submodularity, this implies that in expectation, adding set S”. has marginal value at least 

[J[ ‘ ’ 1 7wp- LEU to set Ui- 1 - Summing up all these inequalities, and using monotonicity of /, we have that: 


nnu m o] > - /(t/i-i)] > X] 


i =1 


2=1 


fc'E[/(S4) -/(C/j-t)] 
2k 


> 


E 

2=1 


fc'(/(^)-E[/([/ m ,)]) 
2k 


We conclude that E[/([7 m /)] is O 


YT=iH.S h ) 


. We also note that E[^=L^a 2] is equal to 


because the expected value of average of some numbers chosen randomly from a set is equal to the average 


of all numbers in the set. Therefore E [f{fJ m i)\ is O 


E7 =1 m) 


y if E " i m /(s<?) is n ^fioPT)), 

have E[/(17 m /)] > Q | sj^-f (OPT) J which concludes the proof of this theorem. So we focus on the case 


a im - o 


yrf( OPT) I, and finish the proof as follows. 


In each machine i for the sake of the analysis, we select |_ Aj items of OPT n T, randomly (if there are 
not these many items in OPT 0 T* take all of them) where 1 < i < m. Define OPT" C OPT be the union 
of these selected items. Since {Ti} 1 f =1 is a random partitioning of all items, we expect f items of OPT to 
be present in set Tj for each i. For each item x € OPT, with 0(1) probability, set does not have more 
than Lf^J items of OPT where i x is the index of the machine that receives x, i.e., x € Tx ■ Therefore, any 
item x € OPT is in set OPT" with 0(1) probability. We conclude that E[/(OPT")] is 0(/(0PT)). Using 
the proof of Lemma l2Al we have that: 


/(opt" n (u ™ =1 s'i)) > /(opt") - £ E A^OPT*) 

t=i xeOPT"n T e \S' e 

Similar to the proofs of Lemmas l2.31 and 12.41 we also have that: 
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E E A(s, OPT 1 ) < ^2 |OPT" nT|\ S'flrg + 

i= 1 xGOPT ,, nTAS; 


E£t/(3) 


£=i 


m 


< 


< 


Eti ^ + YZi f(S' t ) < VFg=i/(M + g=i/W) 


m 

k i i \ 


m 


# +1) Er=i /(5«) 


m 


where the last inequality is implied by the fact that S' f C ,SV, and consequently /(S')) < f(Sf). We 
conclude that: 


/(OPT" n (U^)) > /(OPT")-(W- + 1) £/(£,)/■ 


' 771 


1= 1 


fi(/(OPT))- A /-xo 



/(OPT) = 0(/( OPT)) 


where the equations are implied by the facts that E^Li is o(^/^”/(OPT)), and E[/(OPT")] is 
0(/(0PT)). We note that items of set OPT" n (U/i r S") are not necessarily present in the output sets 
{S"}™ =1 . To finish the proof of this theorem, it suffices to prove that each item in OPT" n S' is present 

in the output set S'" with probability 11 dy for any 1 < i < m. Each item in S\ had marginal value 


at least t/ : to set S' e when it was selected to be added to S'). So there are at most < ISM = y/wk 
items in S’). Since with probability 1/2 the algorithm in machine i outputs k' random items of S’), each item 

in S' f is in the output set with probability at least 


2 Wk 


= Tl I \ K- ). Therefore the intersection of OPT 7 ' 


and the union of output sets, OPT" n (U/Sj 5'/), has expected value Q ( \f^f{ OPT" 0 (U/S 1 S I £))^ = 


Q /(OPT)j. We note that |OPT" n (U ff =1 S'/)\ is at most |OPT"| < |OPT| = k. Therefore we 
have E[/fc(U/S 1 5/)] > E[/(OPT" 0 (U/S 1 5/))] = 0 ((OPT) j which concludes the proof of this 


theorem. 


6 Conclusion 


□ 


The concept of composable core-sets has been introduced recently in the context of distributed and streaming 
algorithms and have been applied to several problems [19, IP). In this paper, we introduced the concept 
of randomized composable core-sets and showed its effectiveness in maximizing submodular functions in 
a distributed manner. While we mainly discuss the cardinality constraint in this paper, we expect that the 
ideas and the proof techniques be applicable to more general packing constraints such as matroid constraints. 
There are several research problems that are interesting to explore in this line of research. 

• For the submodular maximization problem, it remains to find a randomized composable core-set of 
approximation factor 1 — or rule out the possibility of constructing such a core-set. 
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• We discussed how the size and multiplicity of the composable core-set can help improve the approxi¬ 
mation factor of the algorithms. It would be nice to get tight bounds on the approximation factor for 
each range of the size of composable core-sets. Moreover, it would be interesting to study the impact 
of increasing the multiplicity of the core-set on the achievable approximation factors. 

• While we provided a tight result for the small-size composable core-set problem, the achievable ap¬ 
proximation factor is not satisfactory. A natural way to improve this factor is to apply the composable 
core-set idea iteratively, and achieve an improve approximation factor. Even for composable core-sets 
of size k and above, it might be possible to improve the approximation factor by applying such a 
core-set multiple times. This approach leads to several interesting follow-up questions. 

• While randomized composable core-sets are applicable to random-order streaming models, applying 
the proof techniques in this paper may result in stronger approximation factors in pure random-order 
streaming models (compared to the ones presented here). We leave this problem to future research. 

Finally, it would be nice to explore applicability of these ideas on other optimization and graph theoretic 
problems. 
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